My CSV file has [xEF][xBB][xBF] at the start of the first header column name. How can I remove this?

0 votes
asked Apr 17, 2020 by brandon-w-8204 (33,330 points)
When I pick up a csv file there is the following characters at the beginning of my file:  


What are these and how can I remove them?

1 Answer

0 votes

The  [xEF][xBB][xBF] are the BOM (Byte Order Mark) that is included in the file. Java does not expect these characters and so they are included in the message when they should be ignored. 

To remove these we will use a preprocess script.

1. Go to the green source node and check the "Run preprocessing scipt on all messages received":

2. Replace the code with the following script:

var stringValue = new java.lang.String(bytesIn, "WINDOWS-1252");
// If bytesIn begins with a Byte Order Mark at index 0, remove it.
if (stringValue.indexOf("") === 0) {
   bytesOut = java.util.Arrays.copyOfRange(bytesIn, 3, bytesIn.length - 1);
} else {
   bytesOut = bytesIn;

answered Apr 17, 2020 by brandon-w-8204 (33,330 points)