Class JsonRecordPipelineInput
java.lang.Object
com.northconcepts.datapipeline.foundations.core.Bean
com.northconcepts.datapipeline.foundations.core.FoundationObject
com.northconcepts.datapipeline.foundations.pipeline.PipelineObject
com.northconcepts.datapipeline.foundations.pipeline.PipelineInput
com.northconcepts.datapipeline.foundations.pipeline.input.FileSourcePipelineInput
com.northconcepts.datapipeline.foundations.pipeline.input.JsonRecordPipelineInput
- All Implemented Interfaces:
DataExceptionContributor,JsonSerializable,RecordSerializable,XmlSerializable,DataReaderFactory,JavaCodeGenerator,Serializable
Reads a pipeline's source data from a JSON
FileSource using a JsonRecordReader,
which turns each node matching a record break into one nested record.- See Also:
-
Field Summary
Fields inherited from class com.northconcepts.datapipeline.foundations.core.FoundationObject
internalId, internalName, log, TIMESTAMP_FORMATFields inherited from interface com.northconcepts.datapipeline.core.RecordSerializable
SERIALIZED_CLASS_NAME, TYPEFields inherited from interface com.northconcepts.datapipeline.core.XmlSerializable
XML_SERIALIZED_CLASS_NAME -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionaddRecordBreak(String locationPathAsString) Adds a location path, such as/object/users/array/object, whose matching nodes are each read as a record.fromRecord(Record source) Loads this instance's state from a record and returnsthis(for fluid API call chaining).fromXmlElement(Element pipelineInputElement) voidAppends Java source code representing this object to the given builder.getName()Returns a display name for this input, such as the source's name followed by its format.Returns the location paths of the JSON nodes read as records or null if none were added.getRecordBreaks(boolean create) Returns the record break paths, first creating an empty list ifcreateis true and there is none.booleanIndicates if the file's stream is closed when the reader is closed (defaults to true).booleanisDebug()Indicates if the reader logs each parsed node to help find record break paths; not serialized (defaults to false).booleanIndicates if this input can save data lineage on the records it reads (false unless a subclass overrides it).booleanIndicates if JSON decimal numbers are read asBigDecimalinstead of double (defaults to false).setAutoCloseReader(boolean autoCloseReader) Indicates if the file's stream is closed when the reader is closed (defaults to true).setDebug(boolean debug) Indicates if the reader logs each parsed node to help find record break paths; not serialized (defaults to false).setFileSource(FileSource fileSource) setUseBigDecimal(boolean useBigDecimal) Indicates if JSON decimal numbers are read asBigDecimalinstead of double (defaults to false).toRecord()Converts this object's state to a record thatRecordSerializable.fromRecord(Record)can load.toXmlElement(Document document) Returns an element, created withdocument, describing this object; the default implementation throws aDataException.Methods inherited from class com.northconcepts.datapipeline.foundations.pipeline.PipelineInput
getNestedPipelineInput, getPipelineInput, getRootPipelineInput, isSaveLineage, setSaveLineageMethods inherited from class com.northconcepts.datapipeline.foundations.core.FoundationObject
addExceptionProperties, assertValid, assertValid, clone, exception, exception, exception, getInternalId, getInternalName, resetInternalIdMethods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, waitMethods inherited from interface com.northconcepts.datapipeline.core.DataExceptionContributor
addExceptionProperties
-
Constructor Details
-
JsonRecordPipelineInput
public JsonRecordPipelineInput()
-
-
Method Details
-
createDataReader
-
getFileSource
- Specified by:
getFileSourcein classFileSourcePipelineInput
-
setFileSource
- Specified by:
setFileSourcein classFileSourcePipelineInput
-
getName
Description copied from class:PipelineInputReturns a display name for this input, such as the source's name followed by its format.- Overrides:
getNamein classFileSourcePipelineInput
-
isUseBigDecimal
public boolean isUseBigDecimal()Indicates if JSON decimal numbers are read asBigDecimalinstead of double (defaults to false). -
setUseBigDecimal
Indicates if JSON decimal numbers are read asBigDecimalinstead of double (defaults to false). -
getRecordBreaks
Returns the location paths of the JSON nodes read as records or null if none were added. -
getRecordBreaks
Returns the record break paths, first creating an empty list ifcreateis true and there is none. -
addRecordBreak
Adds a location path, such as/object/users/array/object, whose matching nodes are each read as a record. -
isDebug
public boolean isDebug()Indicates if the reader logs each parsed node to help find record break paths; not serialized (defaults to false). -
isLineageSupported
public boolean isLineageSupported()Description copied from class:PipelineInputIndicates if this input can save data lineage on the records it reads (false unless a subclass overrides it).- Overrides:
isLineageSupportedin classPipelineInput
-
setDebug
Indicates if the reader logs each parsed node to help find record break paths; not serialized (defaults to false). -
isAutoCloseReader
public boolean isAutoCloseReader()Indicates if the file's stream is closed when the reader is closed (defaults to true). -
setAutoCloseReader
Indicates if the file's stream is closed when the reader is closed (defaults to true). -
generateJavaCode
Description copied from interface:JavaCodeGeneratorAppends Java source code representing this object to the given builder. -
toRecord
Description copied from interface:RecordSerializableConverts this object's state to a record thatRecordSerializable.fromRecord(Record)can load.- Specified by:
toRecordin interfaceRecordSerializable- Overrides:
toRecordin classPipelineInput
-
fromRecord
Description copied from interface:RecordSerializableLoads this instance's state from a record and returnsthis(for fluid API call chaining). For fluid API call chaining, the overridden method should change the declared return type to its class.- Specified by:
fromRecordin interfaceRecordSerializable- Overrides:
fromRecordin classPipelineInput- Parameters:
source-- Returns:
- this instance.
-
toXmlElement
Description copied from interface:XmlSerializableReturns an element, created withdocument, describing this object; the default implementation throws aDataException.- Specified by:
toXmlElementin interfaceXmlSerializable- Overrides:
toXmlElementin classPipelineInput
-
fromXmlElement
- Specified by:
fromXmlElementin interfaceXmlSerializable- Overrides:
fromXmlElementin classPipelineInput
-