Mule 4: Accumulating and modifying values in DataWeave 2.0

This article is intended for the audience having knowledge on Mule 4 & DataWeave 2.0 and it explains how to accumulate and modify values using dataweave. Aggregating the values in dataweave is a painful task and here in this blog, I will explain different ways of value aggregation in mule 4.

1. Using For Each
2. Using Reduce operation
3. Using Java HashMap class

In this tutorial, I am using Anypoint Studio version 7.14.0 on Mac and going to accumulate the students count who scored more than 80 marks in the respective subjects by using the below sample Json data for all the methods.

---

Using For Each

StudentData holds the Json payload which was mentioned above and I set the collection as #[payload] inside For Each.

Inside For Each, StudentResult transforms each student object into result object by setting the value as 1 if a student scores more than 80 marks in the respective subject, else the value will be 0. Below is the transformation logic

%dw 2.0
output application/json
---
{
	"physics": if(payload.physics >= 80)(1)else(0),
	"maths": if(payload.maths >= 80)(1)else(0),
	"english": if(payload.english >= 80)(1)else(0)
}

accValue variable gets the transformed student object and accumulates the result object by updating the student count with the respective subjects.

%dw 2.0
output application/json
---
if(isEmpty(vars.accValue)){
	(payload)
}
else{
	"physics": vars.accValue.physics + payload.physics,
	"maths": vars.accValue.maths + payload.maths,
	"english": vars.accValue.english + payload.english
}

On 1st iteration, the variable accValue will be empty and we are setting the payload. On subsequent iterations, we are taking the respective subjects value from accValue and adding them with incoming student object value, thereby accumulating the student count on each iteration and below is the final output where we can see, in physics 4 students, in maths 7 students and in english 3 students scored more than 80 marks

{
  "physics": 4,
  "maths": 7,
  "english": 3
}

---

Using Reduce operation

Some of us might not aware of this reduce operator functionality but it will very useful in need of aggregating the values. It iterates over an array of elements like a map operator and stores the result of each iteration, so we can easily accumulates the value and produces the result object with count of students who scored above 80 marks in the respective subjects.

accValue receives the input Json payload, iterates over each object and stores the result of each iteration. Below is the transformation logic

%dw 2.0
var initValue = {
	"physics": 0,
	"maths": 0,
	"english": 0
}
output application/json
---
payload reduce(val, acc=initValue)->{
	"physics": if(val.physics >= 80)(acc.physics + 1) else(acc.physics),
	"maths": if(val.maths >= 80)(acc.maths + 1) else(acc.maths),
	"english": if(val.english >= 80)(acc.english + 1) else(acc.english)
}

Reduce operator takes two arguments, val holds the item in the input array and acc is the accumulator that stores the result of each iteration. I am initialising acc with default value(initValue) as 0 for all subjects. On first iteration, the reduce operator stores the result object in accumulator and on subsequent iterations, those previously stored values can be accessed and it will accumulates the result object with updated student count for all the subjects.

Reference link: https://docs.mulesoft.com/dataweave/2.4/dw-core-functions-reduce

---

Using Java HashMap class

accVar is the initialisation variable with default value as 0 for all subjects. The output mime type must be application/java and it’s holding the below value

output application/java
---
{
	"physics": 0,
	"maths": 0,
	"english": 0
}

accValue is the output variable using Java HashMap class to generate the result object. Below is the transformation logic followed by the details of each function which was mentioned below

%dw 2.0

fun accumulateValues(varName, keyName, accumulate)=
do{
	var setValue = Java::invoke('java.util.HashMap','put(Object,Object)', varName, {
		arg0: keyName,
		arg1: accumulate(Java::invoke(
			'java.util.HashMap',
			'get(Object)',
			varName,
			{arg0: keyName}
		))
	})
	---	Java::invoke('java.util.HashMap','get(Object)',varName, {arg0: keyName})	
}

fun checkCondition(mark, fieldName)=(
	if(mark >= 80)(
		accumulateValues(vars.accVar, fieldName, (val)-> val + 1)
	)
	else(
		accumulateValues(vars.accVar, fieldName, (val)-> val)
	)
)

output application/json
---
(payload map{
	"physics": checkCondition($.physics, "physics"),
	"maths": checkCondition($.maths, "maths"),
	"english": checkCondition($.english, "english")
})[-1]

• accumulateValues is the main function where we are passing three parameters such as varName-the target variable used to accumulate, keyName-the respective field inside that variable and accumulate-the lambda expression used to increment the value.
  1. Java::invoke method is used to access the java modules inside dataweave and you can refer the official mulesoft documentation link on how to use this method https://docs.mulesoft.com/java-module/1.2/java-invoke-method#invoke-methods-with-dataweave
  2. Java::invoke method takes 4 parameters-class, method, target and arguments. In our case,
     • class– java.util.HashMap
     • method– we used 2 different methods from HashMap class- put(Object,Object) & get(Object)
     • target– accVar is the target variable
     • argument– put(Object,Object) takes 2 arguments and get(Object) takes 1 argument
  3. Do operator is composed of header and body, where header used to declare variables & functions and body is the result of the expression. The variables & functions inside do operator can’t be accessed outside the operator and refer this link for more information https://docs.mulesoft.com/dataweave/2.4/dw-operators#control_flow_do
  4. We are calling the function accumulateValues by passing the variable accVar with respective fields-either physics, maths or english and lambda expression. We are updating the value for the respective field in accVar by using put method which takes two arguments-key & value, where arg0 takes the keyName and arg1 takes the last updated value from the accVar by using get method of HashMap class Java::invoke('java.util.HashMap','put(Object,Object)', varName, { arg0: keyName, arg1: accumulate(Java::invoke( 'java.util.HashMap', 'get(Object)', varName, {arg0: keyName} )) })
  5. The result of the expression will be getting the value of the respective field inside accVar by using get method
• checkCondition is another function, which takes 2 parameters- mark and fieldName. While iterating over the payload, we are calling this function with student mark along with subject as fieldName. This function will make a call to that main function and increment the value for the respective field by 1, if the mark is more than 80, or else it won’t do anything. We are passing lambda expression (val)->val + 1 which is used to increment the value.
• In the body expression, I am iterating over the payload and make a call to checkCondition function with mark and subject name, which in turn calls accumulateValues function and update the value based on marks. I used [-1] to get the last object as it holds the final output or else it will return an array of objects processed on each iteration.

Reference link: https://iwconnect.com/how-to-accumulate-and-modify-values-into-variable-using-mule-4-and-dataweave-2/

By following this approach, we can also use other java classes and able to do complex transformations in dataweave.

---

Apart from this above mentioned methods, we can also do transformation like below by using default operators of dataweave, so based on our need we have to chose the correct method which greatly helps to solve our problem.

%dw 2.0
output application/json
---
{
	"physics": sizeOf(payload.physics[?($ >= 80)]),
	"maths": sizeOf(payload.maths[?($ >= 80)]),
	"english": sizeOf(payload.english[?($ >= 80)])
}

Thanks for sparing your time and reading this blog!!