Detect Records in JSON with Header and Details

This example shows how to read a header-plus-details JSON document as one record per detail row, with the header value repeated on every row. The standard TreeDetectionStrategy detects the order_details array as the record break and flags header, a single value outside the array, as a field whose value cascades across records. Passing node.isCascadeFieldValue() as the third argument of addField() is what makes the JsonReader repeat it. The nested product object is flattened into the product_name and order_quantity fields.

Input JSON file

{
    "root": {
        "header": "Orders exported 2025-08-05",
        "body": {
            "order_details": [
                {
                    "product": { "product_name": "Nike Shoes", "order_quantity": 1 },
                    "order_id": "1234",
                    "order_date": "2025-08-05T11:16:47Z"
                },
                {
                    "product": { "product_name": "Clothes", "order_quantity": 2 },
                    "order_id": "456",
                    "order_date": "2025-08-04T04:39:31Z"
                }
            ]
        }
    }
}

Java Code Listing

package com.northconcepts.datapipeline.foundations.examples.tree;

import java.io.File;

import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.core.StreamWriter;
import com.northconcepts.datapipeline.foundations.pipeline.tree.Tree;
import com.northconcepts.datapipeline.foundations.pipeline.tree.detect.TreeDetectionStrategy;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.json.JsonReader;

public class DetectRecordsInJsonWithHeaderAndDetails {

    public static void main(String[] args) {
        File inputFile = new File("example/data/input/tree/header_and_details.json");

        Tree tree = Tree.loadJson(inputFile, TreeDetectionStrategy.standard(true));

        JsonReader reader = new JsonReader(inputFile);
        tree.getAllFields().forEach(node -> reader.addField(node.getFieldName(), node.getXpathExpression(), node.isCascadeFieldValue()));
        tree.getAllRecordBreaks().forEach(node -> reader.addRecordBreak(node.getXpathExpression()));

        DataWriter writer = StreamWriter.newSystemOutWriter();

        Job.run(reader, writer);
    }

}

Code Walkthrough

1. inputFile points at header_and_details.json.
2. Tree.loadJson(inputFile, TreeDetectionStrategy.standard(true)) loads the document into a Tree using the default rules and passes; no extra rule is needed because the details already repeat as an array.
3. A JsonReader is created on the same file and configured from the tree: tree.getAllFields() supplies the name and XPath expression of every detected field to addField(). The third argument, node.isCascadeFieldValue(), tells the reader to repeat a value that spans several records in each of them. header is the cascading field here; the product and order fields belong to one record each.
4. tree.getAllRecordBreaks() supplies the XPath of the array elements to addRecordBreak(), so one record is emitted per order detail.
5. Job.run() streams the records to a StreamWriter on the console.

Console Output

-----------------------------------------------
0 - Record (MODIFIED) {
    0:[header]:STRING=[Orders exported 2025-08-05]:String
    1:[product_name]:STRING=[Nike Shoes]:String
    2:[order_quantity]:LONG=[1]:Long
    3:[order_id]:STRING=[1234]:String
    4:[order_date]:STRING=[2025-08-05T11:16:47Z]:String
}

-----------------------------------------------
1 - Record (MODIFIED) {
    0:[header]:STRING=[Orders exported 2025-08-05]:String
    1:[product_name]:STRING=[Clothes]:String
    2:[order_quantity]:LONG=[2]:Long
    3:[order_id]:STRING=[456]:String
    4:[order_date]:STRING=[2025-08-04T04:39:31Z]:String
}

-----------------------------------------------
2 records