Using JSONPath to Query JSON in OSB

Extracting values from JSON is much easier with JSONPath. Download the full example.

473487986757

Recently we have used Javascript to query and update JSON payloads.

There is a way to extract necessary values from JSON in a much simpler way.

Meet JSONPath.

Javascript, the Imperative Way

In the previuous post, we have written a Javascript code that was navigating the JSON payload in a search of the userid field.

// at this point json variable contains the body

if( !json.hasOwnProperty("userid") ) {
	json.userid = defaultAuthority;
}

// the result variable must be called 'output'
output = json;

The script is relatively simple because the payload has the userid field right on the top:

{
  "aaa":123, 
  "userid":"jane"
}

But what if the value is deep inside of the JSON? We’d have to write long path like this:

var credentials = json.header.metainfo.credentials;
if( !credentials.hasOwnProperty("userid") ) {
	json.header.metainfo.credentials.userid = defaultAuthority;
}

What’s worse, unlike XQuery, a missing field in the Javascript path causes a runtime error, so we may end up with even uglier code:

var header = json.hasOwnProperty("header")? json.header : {};
var metainfo = header.hasOwnProperty("metainfo")? header.metainfo : {};
var credentials = metainfo.hasOwnProperty("credentials")? header.credentials : {};
...

The code quickly becomes not maintainable.

JSONPath, the Declarative Way

Luckily, there is a Javascript library that allows us to use XPath-like notation to query JSON: JSONPath, by Stefan Gössner.

The same deep search for userid, using JSONPath can be written as:


$..userid

Alternatively, if you want to make sure it is the right userid attribute, you can provide the full path – it won’t cause a runtime error is any element of the path doesn’t exist:


$.header.metainfo.credentials.userid

Here’s the syntax match between XPath and JSONPath, per Stefan:

XPath JSONPath Result
/store/book/author $.store.book[*].author The authors of all books in the store
//author $..author All authors
/store/* $.store.* All things in store
/store//price $.store..price The prices of everything in the store
//book[3] $..book[2] The third book
//book[last()] $..book[(@.length-1)]
$..book[-1:]
The last book
//book[position()<3] $..book[0,1]
$..book[:2]
The first two books
//book[isbn] $..book[?(@.isbn)] Filter out all books having no isbn number
//book[price<10] $..book[?(@.price<10)] Filter all books cheaper than 10
//* $..* All Elements in XML document.
All members of JSON structure.

Updated Example

To illustrate the usage of JSONPath, I have created a new example project: JSON-Update-With-JSONPath.

It has the same basic functionality as its Javascript counterpart: check the incoming payload and insert “userid”: “anonymous” if userid is missing.

Unlike the Javascript version, however, it first uses JSONPath to query userid value, and only if it is missing, calls Javascript to insert it.

UpdateWithJSONPath

The query script is as simple as this:

$..userid

I have updated the Java callout code to recognize when the script is actually a JSONPath expression, and execute it in a special way.

I’m checking if the script starts with a dollar sign, and, if yes, I replace it with a call to jsonPath() function (Yes, I know Javascript can have $ too. This is just a demo.). For my tests, I also only use the first found value, ignoring the rest.

    addJSON(engine,bindings);
	    
    // inject and parse the main value
    engine.eval("var json = "+json+";",bindings);
	    
    // JSONPath scripts just have $..ban format
    if( script.startsWith("$") ) {
        addJSONPath(engine,bindings);
		    
        // need to escape quotes in script, but I'm too lazy to do that for a demo
        script = "var output = jsonPath(json,\""+script+"\"); if( output === false ) output = null; else output = output[0];";
        engine.eval(script,bindings);
        return (String)engine.eval("typeof output=='string'||output==null?output:JSON.stringify(output)",bindings);
    } else {
        // execute Javascript
        engine.eval(script,bindings);
        // return the value of output variable as String
        return (String)engine.eval("typeof output=='string'?output:JSON.stringify(output)",bindings);
    }

You can find the complete code in the examples package.

JSONPath Limitations

Read-only

JSONPath library, unlike XQuery, can only retrieve data from JSON, and cannot update it.

Hence, in our case, to insert the userid into JSON we’d still have to use the good old Javascript.

There are many cases, however, where the JSON itself doesn’t have to be updated, but the values from it are used as business rules input.

Javascript-OSB Impedance

JSONPath is a pure Javascript library, and its API is built in the Javascript traditions.

For instance, when the expression matches no nodes, JSONPath returns false. When it matches one or more nodes, it returns an array of of found values.

This design is not very convenient for OSB’s Java callouts. I’d rather prefer null value when nothing is found. It would also be convenient if I could tell JSONPath if I expect only one value, so I do not have to unbox the array of a single element.

No built-in AND clause

JSONPath has a logical-OR operator (|), but surprisingly, it doesn’t have logical-AND. As the result, you cannot write

$..book[?(@.price>5 and @.price<10)]

I hope some Committee eventually gather to complete the specification for JSONPath, adding this and other missing functionality. For the time being, just know its limits.

Vladimir Dyuzhev, author of GenericParallel

About Me

My name is Vladimir Dyuzhev, and I’m the author of GenericParallel, an OSB proxy service for making parallel calls effortlessly.

I’m building SOA enterprise systems for clients large and small for almost 20 years. Most of that time I’m working with BEA (later Oracle) Weblogic platform, including OSB and other SOA systems.

Feel free to contact me if you have a SOA project to design and implement. See my profile on LinkedIn.

I live in Toronto, Ontario, Canada.  canada   Email me at info@genericparallel.com


Tags: , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *

*