Tutorials

The MongoDB Aggregation Pipeline in 5 Minutes

Key Takeaways

  • Each aggregation stage consumes the previous stage’s output: filter documents, group them, then shape or sort the results.
  • $group is a stage; $sum is an accumulator inside it. The sample below totals Jim’s likes to 7 and Joe’s to 3.
  • Put selective filters early when semantics allow, design suitable indexes, and inspect the actual execution plan.
  • The 16 MiB limit applies to individual result documents. Memory thresholds and disk spilling are separate, stage-specific concerns.

The MongoDB aggregation pipeline processes documents through an ordered list of stages to filter, reshape, and summarize collection data. Each stage receives the documents produced by the previous stage—not necessarily the original documents.

Use aggregation for multi-step transformations and reporting. Map-reduce has been deprecated since MongoDB 5.0; focused methods such as countDocuments() and distinct() still make sense when you only need a count or unique field values.

A Quick MongoDB Aggregation Pipeline Example

Open mongosh connected to your MongoDB server. This setup selects a dedicated demo database, resets its posts collection, and inserts three documents with explicit IDs.

db = db.getSiblingDB("stackchief_aggregation_demo");
db.posts.drop();

db.posts.insertMany([
  {
    _id: 1,
    title: "my first post",
    author: "Jim",
    likes: 5,
    tags: ["angular", "react", "javascript"]
  },
  {
    _id: 2,
    title: "my second post",
    author: "Jim",
    likes: 2,
    tags: ["javascript", "react", "vue"]
  },
  {
    _id: 3,
    title: "hello world",
    author: "Joe",
    likes: 3,
    tags: ["vue", "react"]
  }
]);

To calculate total likes for each author:

db.posts.aggregate([
  { $group: { _id: "$author", total_likes: { $sum: "$likes" } } },
  { $sort: { _id: 1 } }
]).toArray();
[
  { _id: "Jim", total_likes: 7 },
  { _id: "Joe", total_likes: 3 }
]

$group produces one document per author, with the author’s name as the output _id. $sum adds the likes within each group. The final sort makes the displayed order predictable because $group does not order its output.

The examples call toArray() to display complete results. It loads the cursor’s results into client memory, so reserve this pattern for bounded results rather than large exports.

How Pipeline Stages Work Together

Add $match before grouping to calculate likes for Joe alone:

db.posts.aggregate([
  { $match: { author: "Joe" } },
  { $group: { _id: "$author", total_likes: { $sum: "$likes" } } }
]).toArray();
[
  { _id: "Joe", total_likes: 3 }
]
posts          $match                 $group              result
3 documents -> author is Joe -> sum likes by author -> Joe: 3 likes
               1 document             1 document

The grouping stage only sees the matching document. More generally, a stage can remove documents, create additional documents, or replace their shape before passing them onward.

Stage Operators, Expressions, and Accumulators

A stage operator, such as $match or $group, names a pipeline step. Inside aggregation expressions, "$author" reads the current document’s author field, while "Jim" is a literal string.

Here, $sum is an accumulator inside $group, not a pipeline stage. { $sum: "$likes" } totals numeric likes; { $sum: 1 } counts the documents entering each group.

MongoDB Aggregation Pipeline Examples

$group: Collect Titles by Author

The $push accumulator collects values into an array. Sort before grouping when array order matters, and sort after grouping when the order of the groups matters.

db.posts.aggregate([
  { $sort: { _id: 1 } },
  { $group: { _id: "$author", titles: { $push: "$title" } } },
  { $sort: { _id: 1 } }
]).toArray();
[
  { _id: "Jim", titles: ["my first post", "my second post"] },
  { _id: "Joe", titles: ["hello world"] }
]

$match: Filter Documents

$match uses query predicates to select documents. This pipeline returns Jim’s two posts without changing their fields.

db.posts.aggregate([
  { $match: { author: "Jim" } },
  { $sort: { _id: 1 } }
]).toArray();
[
  {
    _id: 1, title: "my first post", author: "Jim", likes: 5,
    tags: ["angular", "react", "javascript"]
  },
  {
    _id: 2, title: "my second post", author: "Jim", likes: 2,
    tags: ["javascript", "react", "vue"]
  }
]

$project: Select and Compute Fields

$project controls the output document’s shape. Use 1 to include a field, 0 to exclude it, or an expression to compute a value. The _id field is included by default; excluding it is the usual exception to the rule against mixing field exclusions with inclusions or computed fields.

db.posts.aggregate([
  { $sort: { _id: 1 } },
  {
    $project: {
      _id: 0,
      author: 1,
      abbrev: { $substrCP: ["$title", 0, 3] }
    }
  }
]).toArray();
[
  { author: "Jim", abbrev: "my " },
  { author: "Jim", abbrev: "my " },
  { author: "Joe", abbrev: "hel" }
]

$substrCP counts Unicode code points, starting at a zero-based offset. $substrBytes counts UTF-8 bytes instead. The older $substr operator is deprecated and remains an alias for $substrBytes.

For more inclusion and exclusion examples, see MongoDB projections.

$unwind: Expand an Array

$unwind emits a document for each array element, replacing the array field with that element. This example projects only the post ID and tag so you can see every expanded result clearly.

db.posts.aggregate([
  { $unwind: "$tags" },
  { $sort: { _id: 1, tags: 1 } },
  { $project: { tags: 1 } }
]).toArray();
[
  { _id: 1, tags: "angular" },
  { _id: 1, tags: "javascript" },
  { _id: 1, tags: "react" },
  { _id: 2, tags: "javascript" },
  { _id: 2, tags: "react" },
  { _id: 2, tags: "vue" },
  { _id: 3, tags: "react" },
  { _id: 3, tags: "vue" }
]

The full unwind result contains eight documents: three tags from each Jim post and two from Joe’s post. By default, documents with a missing, null, or empty array field are omitted; use the document form of $unwind with preserveNullAndEmptyArrays: true when you need to retain them.

$sort and $limit: Return the Top Results

$sort uses 1 for ascending order and -1 for descending order. $limit passes at most the specified number of documents onward. Here, the _id tiebreaker makes equal-like results deterministic.

db.posts.aggregate([
  { $match: { author: "Jim" } },
  { $sort: { likes: -1, _id: 1 } },
  { $limit: 1 },
  { $project: { _id: 0, title: 1, likes: 1 } }
]).toArray();
[
  { title: "my first post", likes: 5 }
]

Pipeline Optimization: Indexes and explain()

Put a selective $match early when it filters source fields without changing the intended result. A leading match can use an appropriate index, and a following sort can also use an index when the filter, sort keys, and pipeline shape permit it. A sort after $group cannot use a source-collection index to order the newly grouped results.

For the top-Jim-post pipeline, create a compound index—an index covering multiple fields—with the equality field first and the sort fields afterward. This index can support both the author filter and the requested ordering.

db.posts.createIndex({ author: 1, likes: -1, _id: 1 });

See MongoDB indexing fundamentals and the create-index tutorial for index design and creation.

Inspect the actual plan rather than assuming the index is used. The executionStats mode executes the winning plan to completion and reports its execution statistics.

db.posts.explain("executionStats").aggregate([
  { $match: { author: "Jim" } },
  { $sort: { likes: -1, _id: 1 } },
  { $limit: 1 },
  { $project: { _id: 0, title: 1, likes: 1 } }
]);

Read these fields in the explain results; their nesting varies with the execution engine:

  • winningPlan: inspect the chosen plan for IXSCAN (index scan), COLLSCAN (collection scan), and any explicit sort.
  • totalDocsExamined and totalKeysExamined: documents examined and index entries scanned.
  • nReturned: documents emitted by the reported plan or stage; an upstream count is not necessarily the final aggregation count.
  • executionTimeMillis: planning and execution time, excluding network transfer—not end-to-end application latency.

MongoDB documents optimizations such as moving an adjacent $match ahead of $sort and combining adjacent $sort/$limit stages. These are eligible rewrites, not a promise that every pipeline is efficient. Don’t manually move a filter across a limit: selecting the global top posts and then filtering by author answers a different question from selecting that author’s top posts.

Use the three-document fixture to learn the syntax, not to benchmark. Measure representative data and inspect the plan again after meaningful index or pipeline changes.

Returning or Writing Aggregation Results

aggregate() normally returns a cursor, which lets the client consume results incrementally. Ordinary read pipelines do not modify the source collection; $out and $merge are the stages for writing results to a collection.

db.posts.aggregate([
  { $group: { _id: "$author", total_likes: { $sum: "$likes" } } },
  { $out: "author_likes" }
]).toArray();

db.author_likes.aggregate([
  { $sort: { _id: 1 } }
]).toArray();

The second command returns the same Jim/Joe totals as the first example. Choose $merge instead when you want configurable insertion or updating of matching target documents rather than a whole-collection replacement. Both writing stages must be last in their pipeline.

Performance & Limitations

  • Document size: Each returned or stored BSON document must fit within 16 MiB. This is not a 16 MiB cap on the entire result set or cursor; intermediate pipeline documents may exceed that size.
  • Working memory: MongoDB documents a 100 MB memory threshold for eligible stages such as $group and non-index-backed $sort. It is not a universal cap on every stage or the entire pipeline.
  • Disk spilling: Starting in MongoDB 6.0, allowDiskUseByDefault controls automatic spilling for eligible stages. Per-operation allowDiskUse: true permits spilling, while false prohibits it, overriding the server setting.

For an explicit per-operation override:

db.posts.aggregate(
  [
    { $group: { _id: "$author", total_likes: { $sum: "$likes" } } },
    { $sort: { _id: 1 } }
  ],
  { allowDiskUse: true }
).toArray();

allowDiskUse does not make memory usage unlimited or remove the BSON document-size limit. Some stages, such as $facet, cannot spill to disk. Treat spilling as a fallback, not a substitute for reducing input or fixing indexes.

Common Mistakes

  • Reading fields that no longer exist: After the likes grouping, documents contain _id and total_likes, not the original author, title, or tags fields.
  • Counting posts after expanding tags: In this fixture, summing likes after $unwind counts each post’s likes once per tag, not once per post.
  • Assuming groups or arrays are sorted: Sort grouped output explicitly, and sort input before $push when array order matters.

FAQ

What is a MongoDB aggregation pipeline?

A MongoDB aggregation pipeline is an ordered sequence of stages that filter, reshape, or summarize documents. Each stage receives the previous stage’s output.

How do I use $match and $group together?

Place $match before $group when you want to filter source documents before calculating totals. For example, matching author: "Joe" before grouping by "$author" sums only Joe’s likes.

Is $sum an aggregation stage or an accumulator?

In the examples here, $sum is an accumulator inside $group, not a pipeline stage. { $sum: "$likes" } totals likes, while { $sum: 1 } counts the documents entering each group.

How can I inspect aggregation performance with explain()?

Run db.posts.explain("executionStats").aggregate([...]) in mongosh, replacing [...] with your pipeline stages. Inspect the winning plan, totalDocsExamined, totalKeysExamined, and nReturned; executionStats runs the winning plan to completion.

What are MongoDB aggregation memory limits, and when should I use allowDiskUse?

Eligible stages have a 100 MB memory threshold, with automatic spilling controlled by allowDiskUseByDefault since MongoDB 6.0 and a per-operation allowDiskUse override. Disk spilling does not remove the 16 MiB per-document limit or other stage-specific limits.

Further Reading

Written by Sam Erickson

Software engineer. I write every article on StackChief myself. Author of Mastering the Tech Interview. More about me

Related reading