My CSV file has [xEF][xBB][xBF] at the start of the first header column name. How can I remove this?

Question

When I pick up a csv file there is the following characters at the beginning of my file:  

[xEF][xBB][xBF]

What are these and how can I remove them?

Answer 1

The  [xEF][xBB][xBF] are the BOM (Byte Order Mark) that is included in the file. Java does not expect these characters and so they are included in the message when they should be ignored. 

To remove these we will use a preprocess script.

1. Go to the green source node and check the "Run preprocessing scipt on all messages received":

2. Replace the code with the following script:

var stringValue = new java.lang.String(bytesIn, "WINDOWS-1252");
// If bytesIn begins with a Byte Order Mark at index 0, remove it.
if (stringValue.indexOf("") === 0) {
   bytesOut = java.util.Arrays.copyOfRange(bytesIn, 3, bytesIn.length - 1);
} else {
   bytesOut = bytesIn;
}