Package com.northconcepts.datapipeline.foundations.pipeline.tree.detect
package com.northconcepts.datapipeline.foundations.pipeline.tree.detect
Pluggable algorithms to detect record breaks and fields in a
Tree, including collapsing down
(merging) tree-node-paths where a naming/wildcard pattern can be derived.-
ClassDescriptionDerives a
SiblingSetPatternfrom siblings sharing a fixed prefix/suffix around a varying dynamic segment — for exampleitem-1,item-2,item-3collapse toitem-*.Collapses down (merges) sibling tree-node-paths whose names share a derivable naming/wildcard pattern (seeSiblingSetRule) into a single wildcard node, mirroring what the build-timeNodeNameRules produce for individually dynamic names: the collapsed node is named*, its XPath expression ends innode(), member names become its values, and descendant XPaths collapse the merged segment to*.Classifies date/timestamp names (for example2024-01-15map keys) as dynamic using aDateTimePatternDetector.Marks everyTreeNodethat held at least one text value as a field, assigns its initial field name, and flags values that should cascade across multiple output records.Marks the repeatingTreeNodes that represent one record per instance.Final naming pass: reassigns field names that still collide after the tree structure was transformed, keeping every field name unique.Classifies a single XML element or JSON field name as dynamic data (an identifier, date, or other generated value) rather than a fixed structural name.Cleans up a loadedTreeby merging non-wildcard siblings into an existing wildcard node, adding a synthetic value field under childless wildcard nodes, and marking JSON array elements as record breaks.Classifies names matching a caller-supplied regular expression as dynamic.An immutable naming pattern derived from a set of sibling names — a fixed prefix and suffix around a dynamic middle segment (for exampleitem-*derived fromitem-1,item-2,item-3).Examines a parent's candidate children collectively and derives aSiblingSetPatternwhen their names share a naming/wildcard pattern, or returnsnullwhen no pattern can be derived.The defaultNodeNameRule: treats numeric names, UUID names, and names containing special characters as dynamic, matching.invalid reference
XmlNode#isWildcardCandidate(String)Derives aSiblingSetPatternfrom siblings whose child structures overlap, instead of their names: each candidate's structural signature is the set of its child element and attribute names (looking through JSONobject/arraycontainer nodes), and siblings whose signatures agree aboveminimumSimilarity(Jaccard overlap) form the collapse group.Defines the algorithms used to detect record breaks and fields in aTree: an ordered list ofNodeNameRules that classify dynamic element/field names while the tree is being built, followed by an ordered list ofTreeTransformerpasses that run over the loaded tree.A single pass in aTreeDetectionStrategythat analyzes or restructures aTreeafter it has been loaded from an XML or JSON document.