map, filter, reduce — and Why reduce Is the Only Hard One

The three functions you'll use in every transform, taught properly — including the accumulator, the seed, and what reduce does to an empty array.

Almost every DataWeave script you write is some combination of three functions. map reshapes each element. filter drops the elements you don’t want. reduce collapses many elements into one. Two of them you’ll learn in a minute each; the third repays a little patience, because it’s the one people quietly avoid and then reinvent badly with a var and a lot of concatenation. This post gives reduce the attention it’s owed.

Throughout, the payload is an order:

{
  "orderId": "A-1001",
  "items": [
    { "sku": "PEN-01", "price": 2.5, "qty": 4 },
    { "sku": "PAD-22", "price": 6.0, "qty": 2 },
    { "sku": "CLP-08", "price": 1.0, "qty": 10 }
  ]
}

map: one in, one out

map applies a function to every element and returns a new array of the same length. The lambda receives the item and its index:

%dw 2.0
output application/json
---
payload.items map (item, index) -> {
  line: index + 1,
  sku: item.sku,
  amount: item.price * item.qty
}

Output:

[
  { "line": 1, "sku": "PEN-01", "amount": 10.0 },
  { "line": 2, "sku": "PAD-22", "amount": 12.0 },
  { "line": 3, "sku": "CLP-08", "amount": 10.0 }
]

Nothing was added or removed — three items in, three objects out. When you don’t need the index, drop it, or use the positional $ for the item and $$ for the index:

payload.items map $.price * $.qty

That’s [10.0, 12.0, 10.0]. map never changes the count; it only changes the shape of each element.

filter: keep what passes

filter runs a predicate — a lambda that returns a Boolean — and keeps the elements for which it’s true. The count can shrink; the elements themselves are untouched:

%dw 2.0
output application/json
---
payload.items filter (item) -> item.qty >= 4

Output keeps the pens and the clips and drops the pads. The predicate can use the index as its second parameter too, exactly like map.

Objects have their own version. filterObject keeps key-value pairs rather than array elements — handy for pruning nulls before you write JSON:

%dw 2.0
output application/json
---
{ orderId: "A-1001", coupon: null, note: "gift" } filterObject (value) -> value != null

That drops coupon and leaves { "orderId": "A-1001", "note": "gift" }. Same idea, applied to an object instead of an array.

reduce: many in, one out

reduce walks the array carrying a running result (the accumulator) and returns whatever the accumulator ends up as. The lambda’s parameters are, in this order, the current item and the accumulator:

%dw 2.0
output application/json
---
[10, 20, 30] reduce (item, accumulator) -> accumulator + item

Output: 60. Read the parameter order carefully, because it trips up everyone once: in map and filter the second parameter is the index, but in reduce the second parameter is the accumulator. The positional shorthand makes this genuinely hard to read — reduce ($$ + $) is accumulator + item, and you will not remember that in six months. For reduce, name the parameters. Always.

The seed, and why you almost always want one

You can give the accumulator a starting value, a seed, with a default on its parameter:

[10, 20, 30] reduce (item, acc = 0) -> acc + item

The accumulator starts at 0, and reduce visits all three items: 0 + 10, then + 20, then + 30, giving 60. Now watch what happens when you leave the seed off:

[10, 20, 30] reduce (item, acc) -> acc + item

Also 60 — but by a different route. With no seed, reduce uses the first element as the accumulator’s initial value and starts iterating from the second element. So acc begins at 10, then 10 + 20, then + 30. For a plain sum the answer is the same, which is exactly why the difference stays hidden until it bites.

It bites in two places. First, the empty array:

[] reduce (item, acc) -> acc + item       // null
[] reduce (item, acc = 0) -> acc + item   // 0

With no seed and nothing to seed from, reduce has no accumulator to return, so you get null. A seed gives it a defined answer for the empty case. If an order can have zero line items (and in real feeds it can), an unseeded sum hands you null where you expected 0, and the bug surfaces three transforms downstream.

Second, and more important: the seed sets the accumulator’s type. When the result is a different shape than the elements, the seed is not optional — it’s the only way to tell reduce what it’s building.

Building an object from a list

Turn an array of items into a lookup keyed by SKU. The elements are objects, but the result is a single object, so seed with {}:

%dw 2.0
output application/json
---
payload.items reduce (item, acc = {}) -> acc ++ { (item.sku): item.qty }

Output:

{ "PEN-01": 4, "PAD-22": 2, "CLP-08": 10 }

Each step merges a one-key object onto the accumulator with ++ — the parentheses around item.sku make the key dynamic, evaluating the expression instead of using the literal text. Drop the = {} seed here and reduce would start with the first item ({ sku: ..., price: ..., qty: ... }) as the accumulator, and your output would be wrong in a way no error message warns you about. The seed is load-bearing.

Flattening

Same pattern, arrays instead of objects — seed with [] and concatenate:

[[1, 2], [3, 4], [5]] reduce (item, acc = []) -> acc ++ item

Output: [1, 2, 3, 4, 5]. This works, and it’s worth writing once to feel how general reduce is — but DataWeave ships flatten for exactly this, and you should use it. Which is the real lesson of reduce: it can do anything, so reach for the named function when one exists.

mapObject: map, but for objects

mapObject transforms each key-value pair and returns an object; its lambda gets the value, key, and index:

%dw 2.0
output application/json
---
{ pen: 2.5, pad: 6.0 } mapObject (value, key) -> { (key): value * 100 }

Output: { "pen": 250, "pad": 600 }. It’s to objects what map is to arrays — one pair in, one pair out.

When to reach for reduce

reduce is the engine underneath most of the others — you could write map, filter, groupBy, and sum as reductions, and internally that’s roughly what they are. That’s the reason not to. When a named function says what you mean — summing, grouping, flattening, sorting — it reads better and it’s harder to get wrong than a hand-rolled accumulator. The next post is a tour of those reshaping functions, and most of them are reduce with a good name and a sensible seed already chosen for you.

Keep reduce for the genuinely custom fold — the running total, the object built from a list, the state that no stock function captures. Here’s the one every orders pipeline needs:

%dw 2.0
output application/json
---
{
  orderId: payload.orderId,
  total: payload.items reduce (item, acc = 0) -> acc + (item.price * item.qty)
}

Output: { "orderId": "A-1001", "total": 32.0 }. Seeded at 0, so an empty order gives a total of 0 rather than null — which is the whole point of understanding the seed.

Final thoughts

map and filter are muscle memory within a day. reduce is the one to actually understand, because its two quiet behaviors — first-element-as-seed when you omit one, null on an empty array — are the source of most reduce bugs, and both vanish the moment you make seeding a habit. Seed your accumulator, name your parameters, and prefer a well-named function over a clever fold. Do that and reduce stops being the scary one.

Next: reshaping data — arrays into objects, and back

Comments