Mule 4: Different ways of filtering a payload

Written by

Writing transformation logic is the primary coding aspect in MuleSoft, and Dataweave offers various strategies to implement it. In this tutorial, let’s explore different methods of filtering an payload containing array of objects. Below is the sample product data containing 10 different items, along with various strategies to filter this data based on the category named ‘apparel’.

{
  "products": [
    {
      "id": "GGOEAFKA087499",
      "name": "Android Small Removable Sticker Sheet",
      "description": "Show your Android pride by placing these 8 fun stickers on your technology products or accessories!",
      "price": "2.99",
      "keywords": "Android Small Removable Sticker Sheet, android stickers, sticker sheets, removable sticker sheets, small sticker sheet, android small sticker sheets, Android Sheet",
      "url": "Android+Small+Removable+Sticker+Sheet",
      "category": "accessories",
      "subcategory": "accessories"
    },
    {
      "id": "GGOEAFKA087599",
      "name": "Android Large Removable Sticker Sheet",
      "description": "Show your quirky side by placing these fun Android stickers on your personal belongings.",
      "price": "2.99",
      "keywords": "Android Large Removable Sticker Sheet, android stickers, sticker sheets, removable sticker sheets, large sticker sheet, android large sticker sheets, Android Sheet",
      "url": "Android+Large+Removable+Sticker+Sheet",
      "category": "accessories",
      "subcategory": "accessories"
    },
    {
      "id": "GGOEGEBK094499",
      "name": "Google Bot",
      "description": "This Google Bot can hold multiple poses making it a fun toy for all. Fold the Google Bot back up into a perfect cube when you are done playing.",
      "price": "9.99",
      "keywords": "Google Bot, google bot, bots, natural bots, wood bot, google wood bot",
      "url": "Google+Bot",
      "category": "accessories",
      "subcategory": "accessories"
    },
    {
      "id": "GGOEGFKA086699",
      "name": "Google Emoji Sticker Pack",
      "description": "Who doesn't use emojis? Decorate your space with your current mood!",
      "price": "4.99",
      "keywords": "Google Emoji Sticker Pack, Google sticker pack, emoji sticker pack, google emoji, stickers, pack of sticker, pack of emoji stickers",
      "url": "Google+Emoji+Sticker+Pack+2+sheet",
      "category": "accessories",
      "subcategory": "accessories"
    },
    {
      "id": "GGOEWCKQ085457",
      "name": "Waze Pack of 9 Decal Set",
      "description": "Can't decide which Waze decal to get? We have made that decision easier for you! Now you can purchase a pack of nine Waze Mood Decals!",
      "price": "16.99",
      "keywords": "Waze Pack of 9 Decal Set, decals pack, packs of 9, Waze Packs, Waze Decals, waze, Waze",
      "url": "Waze+Pack+of+9+decal+set",
      "category": "accessories",
      "subcategory": "accessories"
    },
    {
      "id": "GGOEGHPB071610",
      "name": "Google Twill Cap",
      "description": "Classic urban styling distinguishes this Google cap. Retains its shape, even when not being worn.",
      "price": "10.99",
      "keywords": "Google Twill Cap, Google Cap, Google Twill Caps, Google Twill, google cap, google caps, google twill, google twill black cap, google black caps, google caps, black caps, Google Caps",
      "url": "Google+Twill+Cap",
      "category": "apparel",
      "subcategory": "apparel"
    },
    {
      "id": "GGOEGHPJ094299",
      "name": "Google Fold-over Beanie Grey",
      "description": "Keep you ears warm while enjoying a cold winter day with this Google Fold-over Beanie.",
      "price": "9.99",
      "keywords": "Google Fold-over Beanie Grey, gray beanie, grey beanie, Google Beanies, Fold over grey, Google Beanie Grey, Google headgear",
      "url": "Google+Fold+over+beanie+grey",
      "category": "apparel",
      "subcategory": "apparel"
    },
    {
      "id": "GGOEGHPJ094399",
      "name": "Google Pom Beanie Charcoal",
      "description": "Stay stylish and warm this winter season with this Google Pom Beanie.",
      "price": "19.99",
      "keywords": "Google Pom Beanie Charcoal, pom beanie, charcoal pom beanies, Google Beanies, Pom Beanies, charcoal Google pom, beanies, headgear",
      "url": "Google+Pom+Beanie+Charcoal",
      "category": "apparel",
      "subcategory": "apparel"
    },
    {
      "id": "GGOEWXXX0827",
      "name": "Waze Women's Short Sleeve Tee",
      "description": "Made of soft tri-blend jersey fabric, this great t-shirt will help you find your Waze. Made in USA.",
      "price": "18.99",
      "keywords": "Waze Women's Short Sleeve Tee, Waze Short Sleeve Tee, Waze Women's Tees, Waze Women's tee, waze ladies tees, waze ladies tee, waze short sleeve tees, waze short sleeve tee",
      "url": "Waze+Womens+Short+Sleeve+Tee",
      "category": "apparel",
      "subcategory": "apparel"
    },
    {
      "id": "GGOEWXXX0828",
      "name": "Waze Men's Short Sleeve Tee",
      "description": "Made of soft tri-blend jersey fabric, this great t-shirt will help you find your Waze. Made in USA.",
      "price": "18.99",
      "keywords": "Waze Men's Short Sleeve Tee, Waze Short Sleeve Tee, Waze Men's Tees, Waze Men's tee, waze mens tees, waze mens tee, waze short sleeve tees, waze short sleeve tee",
      "url": "Waze+Mens+Short+Sleeve+Tee",
      "category": "apparel",
      "subcategory": "apparel"
    }
  ]
}

Using filter function

The ‘filter’ function is the most commonly used operation for payload filtering. 

Syntax: payload.products filter ($.category == “apparel”)

As you can see, I selected the key ‘products,’ which contains an array of objects, and applied a filter condition on the category named ‘apparel,’ where ‘$’ represents the objects we are iterating over. This is the most straightforward approach, and you can also restructure the syntax as follows:

payload.products filter((Item, Index) -> Item.category == “apparel”)
filter(payload.products, (Item, Index)-> Item.category == “apparel”)
The filter function takes an array as input. If you are working with an object, you should use the filterObject function.

Using conditional operator

This is the simplest syntax accompanied by value selector to filter the payload.

Syntax: payload.products[?($.category == “apparel”)]

The syntax begins with payload, followed by square brackets containing ‘?’ and the filter condition inside parentheses.

Using if condition

Here, we are going to use the map operator along with an if condition to filter the payload. This is one of the most commonly used methods when we want to update any property value based on a condition.

Syntax: flatten(payload.products map [
($) if(($.category) == “apparel”)
])
I take the products array, iterate over it using the map operator, and check if the category has values as ‘apparel’. Two things to notice here are the square brackets followed by the map operator. If you are using parentheses or curly brackets, you must assign a key to the object and then use a value selector to retrieve the object, since I am using square brackets, I can directly mention the object as ‘$’. The if condition returns empty square brackets if it fails. So, finally, using ‘flatten’ helps merge nested arrays by omitting empty subarrays.
 
Suppose if I want to update the price for this particular category, the syntax would be as follows:
import update from dw::util::Values
flatten(payload.products map [
($ update “price” with $+2 ) if(($.category) == “apparel”)
])
First, we need to import the update function from the Values module, and then we can update the respective field. In this example, I simply increase the price by 2 for the categories named ‘apparel’.

Using reduce function

This method comes in handy when we want to recall previously processed values. For example, if you write a program to find the sum of n consecutive numbers, in each iteration, you add up the sum of the previous value with the current number to get the output. This is possible by using the ‘reduce’ operator, as DataWeave does not offer any support for a value assignment inside the ‘map’ or ‘mapObject’ operators.

Syntax: payload.products reduce ((item, accumulator=[]) -> accumulator ++ [
(item) if(item.category == “apparel”)
]
)
The ‘reduce’ expression takes two parameters: ‘item,’ which is the object in the input array, and ‘accumulator,’ which is used to store the result of the lambda expression. The accumulator can be set with an initial value; in this case, I am setting the value as an empty array. In each iteration, I am adding the filtered object with the result produced by the ‘reduce’ expression using the concatenation operator (‘++’).

Apart from the above-mentioned filter methods, you can also use the ‘take’ operator from the Arrays module and execute an if condition by specifying the particular value you want. So, these are the filter methods I am aware of, and if you know any other way of filtering a payload, feel free to leave a comment.

Tags: , ,

Leave a Reply