Class JsonRecordReader
java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataReader
com.northconcepts.datapipeline.json.JsonRecordReader
Obtains records from a JSON stream, turning each object, array or field matched by a record break into one record
that keeps nested objects and arrays as nested records and arrays.
-
Field Summary
Fields inherited from class com.northconcepts.datapipeline.core.DataReader
fieldLineage, recordLineageFields inherited from class com.northconcepts.datapipeline.core.DataEndpoint
lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEYFields inherited from class com.northconcepts.datapipeline.core.Endpoint
BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZEFields inherited from class com.northconcepts.datapipeline.core.DataObject
id, log, name, TIMESTAMP_FORMAT -
Constructor Summary
ConstructorsConstructorDescriptionJsonRecordReader(com.fasterxml.jackson.core.JsonParser reader) JsonRecordReader(File file) JsonRecordReader(Reader reader) -
Method Summary
Modifier and TypeMethodDescriptionaddExceptionProperties(DataException exception) Adds this endpoint's current state to aDataException.protected RecordaddLineage(Record record) Called byDataReader.read()for each record fromDataReader.readImpl()while lineage is saved; the default copiesrecordLineageinto every field along with its original index and name.addRecordBreak(String locationPathAsString) Adds a location path (for example//array/object) matching the JSON objects, arrays or fields to read as records.voidclose()Indicates that this endpoint has finished reading or writing.protected com.fasterxml.jackson.core.JsonFactoryCreates the Jackson factory that theFileandReaderconstructors parse with; the default allows comments, single quotes and unquoted field names.booleanbooleanisDebug()Indicates if each JSON token read is logged at debug level (default is false).booleanIndicates if this reader can capture record and field lineage (false unless overridden by a reader that supports it).protected booleanisRecordBreak(JsonXmlNode node) Returns true if the node matches any location path added withaddRecordBreak(String).booleanIndicates if decimal numbers are read as BigDecimal instead of Double (default is false).booleanReturns whether BigInteger should be used for reading large numeric values.protected JsonXmlNodenewField(JsonXmlNode parentNode, com.fasterxml.jackson.core.JsonToken token, String name, com.fasterxml.jackson.core.JsonLocation tokenLocation) Creates the node for a JSON field name; a scalar value is later stored on it bynewValue(JsonXmlNode, JsonToken, Object).protected JsonXmlNodenewNode(JsonXmlNode parentNode, com.fasterxml.jackson.core.JsonToken token, String name, com.fasterxml.jackson.core.JsonLocation tokenLocation) Creates the node for the start of a JSON object or array (namedobjectorarray).protected JsonXmlNodeStores a scalar value on the current field node, or in a new child node when inside an array, and returns that node.protected booleannextNode()Advances to the next JSON token and updates the current node; returns false at the end of the input.protected RecordreadImpl()Overridden by subclasses to read the next record from thisDataReader.setAutoCloseReader(boolean autoCloseReader) setDebug(boolean debug) Indicates if each JSON token read is logged at debug level (default is false).setUseBigDecimal(boolean useBigDecimal) Indicates if decimal numbers are read as BigDecimal instead of Double (default is false).setUseBigInteger(boolean useBigInteger) Sets whether BigInteger should be used for reading large numeric values.Methods inherited from class com.northconcepts.datapipeline.core.DataReader
available, getBufferSize, getNestedEndpoint, getNestedReader, getReader, getRootEndpoint, getRootReader, isExhausted, isSaveLineage, peek, pop, push, read, setSaveLineage, skipMethods inherited from class com.northconcepts.datapipeline.core.DataEndpoint
decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, open, resetRecordCount, toStringMethods inherited from class com.northconcepts.datapipeline.core.Endpoint
addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription
-
Constructor Details
-
JsonRecordReader
public JsonRecordReader(com.fasterxml.jackson.core.JsonParser reader) -
JsonRecordReader
-
JsonRecordReader
-
-
Method Details
-
createFactory
protected com.fasterxml.jackson.core.JsonFactory createFactory()Creates the Jackson factory that theFileandReaderconstructors parse with; the default allows comments, single quotes and unquoted field names. -
isDebug
public boolean isDebug()Indicates if each JSON token read is logged at debug level (default is false). -
setDebug
Indicates if each JSON token read is logged at debug level (default is false). -
isUseBigDecimal
public boolean isUseBigDecimal()Indicates if decimal numbers are read as BigDecimal instead of Double (default is false). -
setUseBigDecimal
Indicates if decimal numbers are read as BigDecimal instead of Double (default is false). -
isUseBigInteger
public boolean isUseBigInteger()Returns whether BigInteger should be used for reading large numeric values. When enabled, integer values are read as BigInteger instead of Long to handle values larger than Long.MAX_VALUE.- Returns:
- true if BigInteger is used for numeric values, false otherwise (default is false)
-
setUseBigInteger
Sets whether BigInteger should be used for reading large numeric values. When enabled, integer values are read as BigInteger instead of Long to handle values larger than Long.MAX_VALUE.- Parameters:
useBigInteger- true to use BigInteger, false to use Long (default is false)- Returns:
- this JsonRecordReader instance for method chaining
-
addRecordBreak
Adds a location path (for example//array/object) matching the JSON objects, arrays or fields to read as records. -
isRecordBreak
protected boolean isRecordBreak(JsonXmlNode node) Returns true if the node matches any location path added withaddRecordBreak(String). -
close
public void close()Description copied from class:DataEndpointIndicates that this endpoint has finished reading or writing.- Overrides:
closein classDataEndpoint
-
readImpl
Description copied from class:DataReaderOverridden by subclasses to read the next record from thisDataReader. The default implementation ofDataReader.read()now insures that this method will not be called again after it returns anull.If no record is available,
nullwill be returned.Contract for subclasses (see also
docs/authoring/DataReader.md):- Return
nullexactly once to signal end-of-stream. - Do not call this method directly —
DataReader.read()is the template method that wraps exceptions, increments the record count, and tracks lineage. - Do not call
DataEndpoint.incrementRecordCount()here;DataReader.read()already does. - Throw raw exceptions;
DataReader.read()wraps them viaexception(throwable).
- Specified by:
readImplin classDataReader- Throws:
Throwable
- Return
-
nextNode
Advances to the next JSON token and updates the current node; returns false at the end of the input.- Throws:
DataException
-
newNode
protected JsonXmlNode newNode(JsonXmlNode parentNode, com.fasterxml.jackson.core.JsonToken token, String name, com.fasterxml.jackson.core.JsonLocation tokenLocation) Creates the node for the start of a JSON object or array (namedobjectorarray). -
newField
protected JsonXmlNode newField(JsonXmlNode parentNode, com.fasterxml.jackson.core.JsonToken token, String name, com.fasterxml.jackson.core.JsonLocation tokenLocation) Creates the node for a JSON field name; a scalar value is later stored on it bynewValue(JsonXmlNode, JsonToken, Object). -
newValue
protected JsonXmlNode newValue(JsonXmlNode parentNode, com.fasterxml.jackson.core.JsonToken token, Object value) Stores a scalar value on the current field node, or in a new child node when inside an array, and returns that node. -
isLineageSupported
public boolean isLineageSupported()Description copied from class:DataReaderIndicates if this reader can capture record and field lineage (false unless overridden by a reader that supports it).- Overrides:
isLineageSupportedin classDataReader
-
addLineage
Description copied from class:DataReaderCalled byDataReader.read()for each record fromDataReader.readImpl()while lineage is saved; the default copiesrecordLineageinto every field along with its original index and name. Overrides set their source details onrecordLineagefirst and end withsuper.addLineage(record).- Overrides:
addLineagein classDataReader
-
addExceptionProperties
Description copied from class:EndpointAdds this endpoint's current state to aDataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.- Overrides:
addExceptionPropertiesin classDataReader
-
isAutoCloseReader
public boolean isAutoCloseReader() -
setAutoCloseReader
-