Sidebar

Parse Error: Unexpected character ('b' (code 117)): was expecting comma to separate OBJECT entries at...

0 votes
7.6K views
asked Oct 29, 2019 by nader-e-7024 (120 points)

Unexpected character ('b' (code 117)): was expecting comma to separate OBJECT entries
 at [Source: java.io.StringReader@6424dc15; line: 1, column: 33387]

com.qvera.qie.web.exception.MessageModelException: Unexpected character ('b' (code 117)): was expecting comma to separate OBJECT entries...


var json = JSON.parse(source.toString());

the source comes in with mulitple keys/values

No control on what gets sent, there is a double quotes (") before that b in the value

..."patient_name" : "Bobby ack "bob" Kaylab", "patient_id"...

What are my options to handle this?

newbie to qvera, json and java :)

2 Answers

+1 vote

You are right, the string you mentioned is malformed JSON, and won't be parsed by any JSON parser.  The line you mentioned should have escaped the double quotes in the value like this:

..."patient_name" : "Bobby ack \"bob\" Kaylab", "patient_id"...

Without control over the incoming message, you can always wrap the parse in a try catch.  If it fails to parse, you can try to manually correct it and then parse again.  Change your source node to Text, and in your first mapping, convert your message to JSON using the following custom Mapping:

try {
   message = qie.createJSONMessage(source.getNode("/"), "UTF-8");
} catch (err) {
   var malformedJson = source.toString();
   var builder = new java.lang.StringBuilder();
   var commaSplit = StringUtils.splitByWholeSeparator(malformedJson, ',');
   // split on ,
   for (var i = 0; i < commaSplit.length; i++) {
      // split on :
      var colonSplit = StringUtils.splitByWholeSeparatorPreserveAllTokens(commaSplit[i], ':');
      for (var j = 0; j < colonSplit.length; j++) {
         var quoteSplit = StringUtils.splitByWholeSeparatorPreserveAllTokens(colonSplit[j], '"');
         if (quoteSplit.length > 2) {
            // escape any double-quotes between the first and the last double-quotes
            builder.append(quoteSplit[0])
            .append('"')
            .append(StringUtils.join(quoteSplit, "\\\"", 1, quoteSplit.length - 1))
            .append('"')
            .append(quoteSplit[quoteSplit.length - 1]);
         } else {
            builder.append(colonSplit[j]);
         }
         if (j < colonSplit.length - 1) {
            builder.append(":");
         }
      }
      // join with ,
      if (i < commaSplit.length - 1) {
         builder.append(",");
      }
   }
   qie.debug(builder.toString());
   message = qie.parseJSONString(builder.toString(), "UTF-8");

This solution is not perfect.  Your string would still fail if there is unescaped quotes and commas or colons in the value. We are trying to salvage malformed JSON to begin with.

Also, if these messages error, you can always resubmit erred messages and edit the source with the corrected value.

answered Oct 29, 2019 by mike-r-7535 (13,830 points)
edited Oct 29, 2019 by mike-r-7535
commented Oct 29, 2019 by nader-e-7024 (120 points)
thanks mike for your answer

i think that source is being parsed by the engine even before it reaches my code

i created a brand new channel with a source, a mapping and discard nodes
source is a file with that json string in it
the mapping is below

even with the try catch and discarding the message, i'm still getting the same exact error

try {
   var json = JSON.parse(source.toString());
} catch (err) {
   qie.debug("nothing");
   message.discard();
}

I went as far as even removing any code in the mapping node and just leaving this dummy debug line

   qie.debug("nothing");

this is the actual error message i'm getting

Unexpected character ('b' (code 117)): was expecting comma to separate OBJECT entries
at [Source: java.io.StringReader@efbf5c; line: 1, column: 33387]
commented Oct 29, 2019 by mike-r-7535 (13,830 points)
Okay, with the source type being JSON, it needs to be able to parse JSON in order to even make it to a mapping node.  You have two options...

1. Change your source type to Text.  Then in your first mapping, try to parse it and save the new message as JSON.
2. On the source node, on the configuration tab, at the bottom, you can check the box to "Run preprocessing script on" and select "only messages fail to parse".
 - In your preprocess script you will need to set bytesOut to the corrected JSON string.
0 votes

Here is the code you can use in a PreProcess script:

 

//Preprocess script THIS GOES IN PREPROCSS SCRIPT
var preMessageString = new java.lang.String(bytesIn, 'UTF-8');

try {
   qie.parseJSONString(preMessageString.getNode("/"), "UTF-8");
} catch (err) {
   var malformedJson = preMessageString;
   var builder = new java.lang.StringBuilder();
   var commaSplit = StringUtils.splitByWholeSeparator(malformedJson, ',');
   // split on ,
   for (var i = 0; i < commaSplit.length; i++) {
      // split on :
      var colonSplit = StringUtils.splitByWholeSeparatorPreserveAllTokens(commaSplit[i], ':');
      for (var j = 0; j < colonSplit.length; j++) {
         var quoteSplit = StringUtils.splitByWholeSeparatorPreserveAllTokens(colonSplit[j], '"');
         if (quoteSplit.length > 2) {
            // escape any double-quotes between the first and the last double-quotes
            builder.append(quoteSplit[0])
            .append('"')
            .append(StringUtils.join(quoteSplit, "\\\"", 1, quoteSplit.length - 1))
            .append('"')
            .append(quoteSplit[quoteSplit.length - 1]);
         } else {
            builder.append(colonSplit[j]);
         }
         if (j < colonSplit.length - 1) {
            builder.append(":");
         }
      }
      // join with ,
      if (i < commaSplit.length - 1) {
         builder.append(",");
      }
   }
   qie.debug(builder.toString());
   preMessageString = builder.toString();

}

try {
   qie.parseJSONString(preMessageString);
   bytesOut = preMessageString.getBytes();
} catch (err) {
   qie.debug('error with parsing JSON string: ' + err);
}
if (bytesOut === null) {  //When null, the message is discarded and no further processing is done
   //todo: when bytesOut = null, set responseBytes to acknowledge the receipt of discarded messages as needed
   responseBytes = new java.lang.String('Replace with valid acknowledgement').getBytes();
}

answered Oct 30, 2019 by brandon-w-8204 (33,170 points)
...