Sidebar

CSV is quoting fields even though "quote values" is set to false

0 votes
239 views
asked Sep 2, 2022 by nathan-c-4426 (540 points)

We have a channel that does a qie.evaluateTemplate() containing a node path that is not always valid (sometimes that path is not in the source message) and output the results to a CSV.  We notice that the portion of the CSV containing the invalid node tag is always quoted, even if though "quote values" is set to false.

Our code looks like this:

message:

 


{
  "test1": "test",
  "test2": "test2"
}

channel code:

 

message = qie.createCSVMessage('', false, '"', ',', 'UTF-8', false);
var template = "Test 1 is {/test1},Test 2 is {/test2},Test 3 is {/test3}";
var templateOutput = qie.evaluateTemplate(template);
qie.debug(templateOutput);
message.setNode('/', templateOutput);

after processing this is the CSV we get:


Test 1 is test,Test 2 is test2,"Test 3 is "


Why are we getting a quotes when we specifically asked NOT to have fields quoted?

2 Answers

0 votes
Leading and trailing spaces require quotes.

 

Here is a KB that shows how to trim the whitespace for this specific scenario.

https://www.qvera.com/kb/index.php/2294
answered Sep 2, 2022 by michael-h-5027 (14,390 points)
0 votes

QIE uses a 3rd party library to correctly parse CSV.  This library has some modes that it can be configured for.  Currently we support 2 modes:

QuoteMode.ALL

or 

QuoteMode.MINIMAL

We use ALL in the case where quotes around everything is the desired result, and we use "MINIMAL" when quotes are not desired.  MINIMAL usually results in no-quotes, however certain situations can still trigger quotes to be placed around fields.

Quotes are added when the cell ends with a space or any character with a lower numeric value i.e. tab or newline.

So in the case above because 'Test 3 is ' ends with a space when no value is provided, quotes are placed around the field. 

There are multiple work-arounds for when this occurs.  Heres a few for the example above:

1. Provide a dummy value 


// validate values aren't empty:
if (!message.checkNodeExists('/test3') || StringUtils.isEmpty(message.getNode('/test3'))) {
   message.setNode('/test3','N/A');
}
var template = "Test 1 is {m:/test1},Test 2 is {m:/test2},Test 3 is {m:/test3}";
var templateOutput = qie.evaluateTemplate(template);
message = qie.createCSVMessage('', false, '"', ',', 'UTF-8', false);
qie.debug(templateOutput);
message.setNode('/', templateOutput);


qie.warn(message.getNode("/"));

 

2. Replace placeholder


var template = "Test 1 is {m:/test1}|**|,Test 2 is {m:/test2}|**|,Test 3 is {m:/test3}|**|";
var templateOutput = ("" + qie.evaluateTemplate(template)).replace(/\s*\|\*\*\|/g, '');
message = qie.createCSVMessage('', false, '"', ',', 'UTF-8', false);
message.setNode('/', templateOutput);

qie.warn(message.getNode("/"));
answered Sep 2, 2022 by nathan-c-4426 (540 points)
...