1. Home
  2. Documentation
  3. User Guide
  4. JSONata

JSONata

JSONata is a lightweight query and transformation language for JSON-shaped data. DataPipeline includes a JSONata engine that evaluates expressions directly against its own ValueNode tree (records, arrays, and single values), so one expression can select, filter, reshape, or summarize data that arrived as JSON, XML, Avro, Parquet, CSV, database rows, or any other format DataPipeline reads. This page covers the Java API; the language itself is summarized on the JSONata Expressions page and documented in full at docs.jsonata.org.

Add the JSONata Dependency

JSONata ships as the integrations-jsonata add-on. Add it alongside your DataPipeline edition (see Getting Started).

Maven 
<dependency>
  <groupId>com.northconcepts</groupId>
  <artifactId>northconcepts-datapipeline-integrations-jsonata</artifactId>
  <version>11.0.0</version>
</dependency>
Gradle 
implementation 'com.northconcepts:northconcepts-datapipeline-integrations-jsonata:11.0.0'

Compile and Evaluate Expressions

The static Jsonata.jsonata(String) method parses an expression once and returns a reusable Jsonata object (most code imports it statically). Calling evaluate(ValueNode) runs the expression against your data and returns the result as another ValueNode. Syntax errors are reported when the expression is compiled, evaluation errors when it runs, and a compiled expression can be evaluated as many times as you like, from any thread.

Run against the donut catalog used by the Query JSON using JSONata example, this prints:

Note the $. in front of the object constructor. An object constructor is evaluated once against its context, so on an array input a bare {'id': id, 'name': name} produces a single object whose fields are arrays ({"id":["0001","0002"],"name":["Cake","Raised"]}); prefixing it with $., or with any path that selects the items, builds one object per item.

Query Any Data Format

Because JSONata evaluates against a ValueNode, and every Record, ArrayValue, and SingleValue is one, there are several ways to put data in front of an expression:

  • JSON and XML documents: Record.fromJson(reader) and Record.fromXml(reader) load a whole document into memory as a tree. A JSON document is queried from its root (name, $[name='Raised']); an XML document from its root element (document.record.name).
  • Any DataReader: collect the records with new RecordList(reader).toRecord(), which wraps them in a single record under a records array, then query with records.name, $count(records), and so on. The Avro and Parquet examples below work this way, and it applies equally to CSV, Excel, JDBC, or any other reader.
  • Plain Java values: ValueNode.from(Object) converts lists, maps, and scalars.

Working with Results

The result takes whatever shape the expression produced: a record for an object constructor or a filter that matches one item, an array for a sequence of matches, or a single value for a count, a sum, or a scalar field. Use the ValueNode helpers to work with it in Java:

  • asRecord(), asArrayValue(), and asSingleValue() give the typed node, so asSingleValue().getValueAsInteger() reads a count and asRecord().getFieldValueAsString("name", null) reads a field.
  • toJson() and toXml() serialize it; toString() uses DataPipeline's record format, which is handy for debugging.
  • An array of records goes back into a pipeline through new RecordList().fromArrayValue(result.asArrayValue()) and a MemoryReader; a single record through new MemoryReader(new RecordList(result.asRecord())).

Variables and Custom Functions

Expressions can use variables and functions supplied from Java. assign(name, value) binds a variable on the compiled expression for every evaluation, and registerFunction(name, lambda) adds a function callable as $name(...). Lambdas taking zero, one, or two arguments are accepted directly; for more arguments, or to have JSONata validate the argument types, pass a JFunction with a signature string. To bind values for one evaluation only, create a Frame with createFrame(), bind(...) the values, and pass it as the second argument to evaluate.

JSONata Examples

See also the JSONata Expressions language summary and the JSONata Javadocs.

< Expression Language
Contents
SQL Generators >
Mobile Analytics