In this article, I will provide you with a detailed guide on how to efficiently read and write CSV files in Mule 4, covering both scenarios of handling files with and without mule connectors.
Using transform component
Let’s explore how to read and write a file using DataWeave in the Transform component.
Read a file
Dataweave offers a ‘readUrl‘ function that makes it easy to read a file from the classpath (src/main/resources)
Syntax: readUrl(url: String, contentType: String = “application/dw”, readerProperties: Object = {}): Any
Take a transform component and add the below code to read the file
%dw 2.0
output application/json
---
readUrl("classpath://customers-100.csv", "application/csv", {"separator": ","})
- The file customers-100.csv is directly placed under the directory src/main/resources. If there are any folders inside that directory, the syntax would be as follows:
classpath://yourFolder/filename
- The contentType will be set to application/csv since we are reading a CSV file.
- Reader properties should be enclosed within curly braces and to learn more about available properties refer to the mulesoft documentation. For the sake of example, I am using separator property with the value as comma. However, if the file uses a different separator, you can pass the respective value.
Write a file
Dataweave doesn’t provide direct functions (to the best of my knowledge) like ‘readUrl’ for writing a file, but you can utilize Java code within Dataweave to achieve file writing without relying on the file connector.
I created a java class named WriteCsv under the directory src/main/java and added the following code
package file;
import java.io.FileWriter;
public class WriteCsv {
public static void writeFile(String data) {
try {
// Creates a FileWriter
FileWriter file = new FileWriter("/Users/*****/AnypointStudio/studio-workspace/csv/src/main/resources/output.csv");
// Writes the string to the file
file.write(data);
// Closes the writer
file.close();
} catch (Exception e) {
e.getStackTrace();
}
}
} Above is the simple java code, where I created a static function named writeFile that takes String value and generates a file named output.csv in the respective path. Let’s see how to invoke this function using dataweave
%dw 2.0
import java!file::WriteCsv
var customer = write(payload, "application/csv", {"quoteValues":true})
output application/java
---
WriteCsv::writeFile(customer) -
import java!file::WriteCsv
- where file is the package name and WriteCsv is the class name.
- For the payload, I take the first 50 customer details and use the write function to create an output string, storing it in the customer variable. I use a writer property called quoteValues because some values have commas. In the file, you might encounter company values like ‘Dominguez, Mcmillan, and Donovan.‘ Using a comma as the separator could break these values, so I use the ‘quoteValues’ property to protect the actual values and refer to the mulesoft documentation for more writer properties.
- writeFile is a function that accepts a string value, and I pass the ‘customer’ variable to this function. Alternatively, you can directly use the function without specifying the class name by importing the specific method, and the syntax is as follows:
-
import writeFile from java!file::WriteCsv
—
writeFile(customer)
-
Note: If you don’t see the file created in the respective path, simply refresh the project, and it will be displayed.
Using file connector
Now, let’s explore how to read and write a file using the file connector. The connector’s main features as per the mulesoft documentation include:
- The ability to read files or fully list directory contents on demand
- Support for common file operations such as creating directories, reading, writing, copying, moving, renaming, deleting and listing files
- Support for locking files
- File matching functionality
Read a file
Drag the read component from the imported file connector and add it to your flow.
There are two ways to read a file: either by providing the full path to the file or a relative path. To read the file using the first approach, you specify the respective path. In the second approach, you use a connection object (i.e., working directory) along with the file path. The advantage of the second method is that you can reuse the connection object if multiple read/write operations occur in that directory.
Create the file configuration by specifying the working directory from which we will read a file.
Select the MIME type as 'application/csv' and include the necessary parameters. In this example, I choose to skip the first line containing header columns and set the 'header' property to 'false' to create a file without headers. Leave the remaining configuration settings at their default values and run the code; you will receive the output.
Write a file
Drag the write component and integrate it into your flow. Similar to the read operation, you can write a file either by specifying the full path or a relative path.
After specifying the path, you need to provide content to write a file. In this example, I utilize the 'quoteValues' property, remove the 'Phone' columns, and write the payload into the file named 'output.csv'. Following are the writing modes supported by the file connector
| Modes | File Exists | File not exists |
|---|---|---|
|
OVERWRITE |
Overwrite the content completely |
Creates a file |
|
APPEND |
Add the content to the end |
Creates a file |
|
CREATE_NEW |
Throws an error |
Creates a file |
I prefer using Mule connectors whenever possible, but there are instances where we may need to proceed without connectors. For example, I used the FTP connector and needed to send site commands, but I didn’t find any function in that connector for such operations. Depending on the situation, it’s essential to choose the approach that best fits the requirements. Thanks for reading! Feel free to leave a comment if you find any mistakes.