Collapse Numbered Siblings in JSON

This example shows how to read numbered siblings such as item-1, item-2 and item-3 as a repeating list of records. Calling enableSiblingPatternCollapse() on the TreeDetectionStrategy inserts a CollapseSiblingPatterns pass ahead of the detection passes; it merges siblings whose names follow a derivable pattern into one node, so each item is read as its own record. The original element name is preserved in the order_item field. The XML version of this example is Collapse Numbered Siblings in XML.

Input JSON file

{
    "order": {
        "item-1": { "sku": "A100", "qty": 2 },
        "item-2": { "sku": "B200", "qty": 1 },
        "item-3": { "sku": "C300", "qty": 5 },
        "summary": { "total": 8 }
    }
}

Java Code Listing

package com.northconcepts.datapipeline.foundations.examples.tree;

import java.io.File;

import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.core.StreamWriter;
import com.northconcepts.datapipeline.foundations.pipeline.tree.Tree;
import com.northconcepts.datapipeline.foundations.pipeline.tree.detect.TreeDetectionStrategy;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.json.JsonReader;

public class CollapseNumberedSiblingsInJson {

    public static void main(String[] args) {
        File inputFile = new File("example/data/input/tree/numbered_items.json");

        TreeDetectionStrategy strategy = TreeDetectionStrategy.standard(true)
                .enableSiblingPatternCollapse();

        Tree tree = Tree.loadJson(inputFile, strategy);

        JsonReader reader = new JsonReader(inputFile);
        tree.getAllFields().forEach(node -> reader.addField(node.getFieldName(), node.getXpathExpression()));
        tree.getAllRecordBreaks().forEach(node -> reader.addRecordBreak(node.getXpathExpression()));

        DataWriter writer = StreamWriter.newSystemOutWriter();

        Job.run(reader, writer);
    }

}

Code Walkthrough

1. inputFile points at numbered_items.json.
2. TreeDetectionStrategy.standard(true) returns the default detection strategy; the true argument turns on tree optimization, which merges dynamically named nodes into a single wildcard node while the tree is being built. enableSiblingPatternCollapse() adds the sibling pattern pass.
3. Tree.loadJson(inputFile, strategy) loads the document into a Tree. The collapse pass merges item-1, item-2 and item-3 into one wildcard node, and the detection passes then mark that node as the record break with sku and qty as its fields. The summary object sits beside the items under order, so it is read as a fourth record whose total is filled and whose item fields are null; filter it out downstream if only the items are wanted.
4. A JsonReader is created on the same file and configured from the tree: tree.getAllFields() supplies the name and XPath expression of every detected field to addField(). order_item holds the element name, sku and qty the item values and total the summary value.
5. tree.getAllRecordBreaks() supplies the XPath of the record node to addRecordBreak().
6. Job.run() streams the records to a StreamWriter on the console.

Console Output

-----------------------------------------------
0 - Record (MODIFIED) {
    0:[order_item]:STRING=[item-1]:String
    1:[sku]:STRING=[A100]:String
    2:[qty]:LONG=[2]:Long
    3:[total]:STRING=[null]
}

-----------------------------------------------
1 - Record (MODIFIED) {
    0:[order_item]:STRING=[item-2]:String
    1:[sku]:STRING=[B200]:String
    2:[qty]:LONG=[1]:Long
    3:[total]:STRING=[null]
}

-----------------------------------------------
2 - Record (MODIFIED) {
    0:[order_item]:STRING=[item-3]:String
    1:[sku]:STRING=[C300]:String
    2:[qty]:LONG=[5]:Long
    3:[total]:STRING=[null]
}

-----------------------------------------------
3 - Record (MODIFIED) {
    0:[order_item]:STRING=[summary]:String
    1:[sku]:STRING=[null]
    2:[qty]:STRING=[null]
    3:[total]:LONG=[8]:Long
}

-----------------------------------------------
4 records