Using JSONPath to Query JSON in OSB

Extracting values from JSON is much easier with JSONPath. Download the full example.

Recently we have used Javascript to query and update JSON payloads.

There is a way to extract necessary values from JSON in a much simpler way.

Meet JSONPath.

See other posts about OSB & JSON:
Using Javascript to Inspect & Modify JSON Payload
JSON Proxies: Inspecting & Modifying the Payload (using conversion to XML)
Why JSON Does Help Direct Proxy Performance
How To Build a JSON Pass-Through Proxy in OSB
OSB and JSON Proxies: Gathering Statistics

Javascript, the Imperative Way

In the previuous post, we have written a Javascript code that was navigating the JSON payload in a search of the userid field.

// at this point json variable contains the body

if( !json.hasOwnProperty("userid") ) {
    json.userid = defaultAuthority;
}

// the result variable must be called 'output'
output = json;

The script is relatively simple because the payload has the userid field right on the top:

{
  "aaa":123, 
  "userid":"jane"
}

But what if the value is deep inside of the JSON? We’d have to write long path like this:

var credentials = json.header.metainfo.credentials;
if( !credentials.hasOwnProperty("userid") ) {
    json.header.metainfo.credentials.userid = defaultAuthority;
}

What’s worse, unlike XQuery, a missing field in the Javascript path causes a runtime error, so we may end up with even uglier code:

var header = json.hasOwnProperty("header")? json.header : {};
var metainfo = header.hasOwnProperty("metainfo")? header.metainfo : {};
var credentials = metainfo.hasOwnProperty("credentials")? header.credentials : {};
...

The code quickly becomes not maintainable.

JSONPath, the Declarative Way

Luckily, there is a Javascript library that allows us to use XPath-like notation to query JSON: JSONPath, by Stefan Gössner.

The same deep search for userid, using JSONPath can be written as:

$..userid

Alternatively, if you want to make sure it is the right userid attribute, you can provide the full path - it won’t cause a runtime error is any element of the path doesn’t exist:

$.header.metainfo.credentials.userid

Here’s the syntax match between XPath and JSONPath, per Stefan:

Updated Example

To illustrate the usage of JSONPath, I have created a new example project: JSON-Update-With-JSONPath.

It has the same basic functionality as its Javascript counterpart: check the incoming payload and insert “userid”: “anonymous” if userid is missing.

Unlike the Javascript version, however, it first uses JSONPath to query userid value, and only if it is missing, calls Javascript to insert it.

The query script is as simple as this:

$..userid

I have updated the Java callout code to recognize when the script is actually a JSONPath expression, and execute it in a special way.

I’m checking if the script starts with a dollar sign, and, if yes, I replace it with a call to jsonPath() function (Yes, I know Javascript can have $ too. This is just a demo.). For my tests, I also only use the first found value, ignoring the rest.

    addJSON(engine,bindings);

    // inject and parse the main value
    engine.eval("var json = "+json+";",bindings);

    // JSONPath scripts just have $..ban format
    if( script.startsWith("$") ) {
        addJSONPath(engine,bindings);

        // need to escape quotes in script, but I'm too lazy to do that for a demo
        script = "var output = jsonPath(json,\""+script+"\"); if( output === false ) output = null; else output = output[0];";
        engine.eval(script,bindings);
        return (String)engine.eval("typeof output=='string'||output==null?output:JSON.stringify(output)",bindings);
    } else {
        // execute Javascript
        engine.eval(script,bindings);
        // return the value of output variable as String
        return (String)engine.eval("typeof output=='string'?output:JSON.stringify(output)",bindings);
    }

You can find the complete code in the examples package.

JSONPath Limitations

Read-only

JSONPath library, unlike XQuery, can only retrieve data from JSON, and cannot update it.

Hence, in our case, to insert the userid into JSON we’d still have to use the good old Javascript.

There are many cases, however, where the JSON itself doesn’t have to be updated, but the values from it are used as business rules input.

Javascript-OSB Impedance

JSONPath is a pure Javascript library, and its API is built in the Javascript traditions.

For instance, when the expression matches no nodes, JSONPath returns false. When it matches one or more nodes, it returns an array of of found values.

This design is not very convenient for OSB’s Java callouts. I’d rather prefer null value when nothing is found. It would also be convenient if I could tell JSONPath if I expect only one value, so I do not have to unbox the array of a single element.

No built-in AND clause

JSONPath has a logical-OR operator (|), but surprisingly, it doesn’t have logical-AND. As the result, you cannot write

$..book[?(@.price>5 and @.price<10)]

I hope some Committee eventually gather to complete the specification for JSONPath, adding this and other missing functionality. For the time being, just know its limits.