Class XmlPipelineInput
java.lang.Object
com.northconcepts.datapipeline.foundations.core.Bean
com.northconcepts.datapipeline.foundations.core.FoundationObject
com.northconcepts.datapipeline.foundations.pipeline.PipelineObject
com.northconcepts.datapipeline.foundations.pipeline.PipelineInput
com.northconcepts.datapipeline.foundations.pipeline.input.FileSourcePipelineInput
com.northconcepts.datapipeline.foundations.pipeline.input.XmlPipelineInput
- All Implemented Interfaces:
DataExceptionContributor,JsonSerializable,RecordSerializable,XmlSerializable,DataReaderFactory,JavaCodeGenerator,Serializable
- Direct Known Subclasses:
JsonPipelineInput
Reads a pipeline's source data from an XML
FileSource using an XmlReader,
which selects fields and record breaks by location path (a subset of XPath).- See Also:
-
Field Summary
Fields inherited from class com.northconcepts.datapipeline.foundations.core.FoundationObject
internalId, internalName, log, TIMESTAMP_FORMATFields inherited from interface com.northconcepts.datapipeline.core.RecordSerializable
SERIALIZED_CLASS_NAME, TYPEFields inherited from interface com.northconcepts.datapipeline.core.XmlSerializable
XML_SERIALIZED_CLASS_NAME -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionAdds a field read from the nodes matching a location path such as/teachers/teacher/@memberid.Adds a field; withcascadeValuesits last value is carried into later records until the path matches again.Adds a field whose last value is carried into later records until a node matchingcascadeResetLocationPathclears it; a blank reset path turns cascading off.addRecordBreak(String locationPathAsString) Adds a location path, such as/teachers/teacher, whose matching nodes delimit records.protected XmlReaderCreates the reader for the file's stream;createDataReader()then applies this input's settings.fromRecord(Record source) Loads this instance's state from a record and returnsthis(for fluid API call chaining).fromXmlElement(Element pipelineInputElement) voidAppends Java source code representing this object to the given builder.protected voidgenerateJavaCodeImpl(JavaCodeBuilder code, CodeWriter writer) Writes the reader declaration (and any format-specific setters), leaving the writer indented for the common setters.Returns how a field matched more than once within a record is stored (defaults toXmlReader.DuplicateFieldPolicy.USE_LAST_VALUE).Returns the fields to read; null or empty if none were added.getFields(boolean create) Returns the fields to read, first creating an empty list ifcreateis true and there is none.getName()Returns a display name for this input, such as the source's name followed by its format.Returns the location paths that delimit records or null if none were added.getRecordBreaks(boolean create) Returns the record break paths, first creating an empty list ifcreateis true and there is none.booleanIndicates if the file's stream is closed when the reader is closed (defaults to true).booleanisDebug()Indicates if the reader logs each parsed node to help find field and record break paths; not serialized (defaults to false).booleanIndicates if this input can save data lineage on the records it reads (false unless a subclass overrides it).setAutoCloseReader(boolean autoCloseReader) Indicates if the file's stream is closed when the reader is closed (defaults to true).setDebug(boolean debug) Indicates if the reader logs each parsed node to help find field and record break paths; not serialized (defaults to false).setDuplicateFieldPolicy(XmlReader.DuplicateFieldPolicy duplicateFieldPolicy) Sets how a field matched more than once within a record is stored (defaults toXmlReader.DuplicateFieldPolicy.USE_LAST_VALUE).setFileSource(FileSource fileSource) toRecord()Converts this object's state to a record thatRecordSerializable.fromRecord(Record)can load.toXmlElement(Document document) Returns an element, created withdocument, describing this object; the default implementation throws aDataException.Methods inherited from class com.northconcepts.datapipeline.foundations.pipeline.PipelineInput
getNestedPipelineInput, getPipelineInput, getRootPipelineInput, isSaveLineage, setSaveLineageMethods inherited from class com.northconcepts.datapipeline.foundations.core.FoundationObject
addExceptionProperties, assertValid, assertValid, clone, exception, exception, exception, getInternalId, getInternalName, resetInternalIdMethods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, waitMethods inherited from interface com.northconcepts.datapipeline.core.DataExceptionContributor
addExceptionProperties
-
Constructor Details
-
XmlPipelineInput
public XmlPipelineInput()
-
-
Method Details
-
createDataReader
-
createDataReaderImpl
Creates the reader for the file's stream;createDataReader()then applies this input's settings. Subclasses override it to read other formats. -
getFileSource
- Specified by:
getFileSourcein classFileSourcePipelineInput
-
setFileSource
- Specified by:
setFileSourcein classFileSourcePipelineInput
-
getName
Description copied from class:PipelineInputReturns a display name for this input, such as the source's name followed by its format.- Overrides:
getNamein classFileSourcePipelineInput
-
getFields
Returns the fields to read; null or empty if none were added. -
getFields
Returns the fields to read, first creating an empty list ifcreateis true and there is none. -
addField
-
addField
Adds a field read from the nodes matching a location path such as/teachers/teacher/@memberid. -
addField
Adds a field; withcascadeValuesits last value is carried into later records until the path matches again. -
addField
public XmlPipelineInput addField(String name, String locationPathAsString, String cascadeResetLocationPath) Adds a field whose last value is carried into later records until a node matchingcascadeResetLocationPathclears it; a blank reset path turns cascading off. -
getRecordBreaks
Returns the location paths that delimit records or null if none were added. -
getRecordBreaks
Returns the record break paths, first creating an empty list ifcreateis true and there is none. -
addRecordBreak
Adds a location path, such as/teachers/teacher, whose matching nodes delimit records. -
getDuplicateFieldPolicy
Returns how a field matched more than once within a record is stored (defaults toXmlReader.DuplicateFieldPolicy.USE_LAST_VALUE). -
setDuplicateFieldPolicy
public XmlPipelineInput setDuplicateFieldPolicy(XmlReader.DuplicateFieldPolicy duplicateFieldPolicy) Sets how a field matched more than once within a record is stored (defaults toXmlReader.DuplicateFieldPolicy.USE_LAST_VALUE). -
isDebug
public boolean isDebug()Indicates if the reader logs each parsed node to help find field and record break paths; not serialized (defaults to false). -
setDebug
Indicates if the reader logs each parsed node to help find field and record break paths; not serialized (defaults to false). -
isAutoCloseReader
public boolean isAutoCloseReader()Indicates if the file's stream is closed when the reader is closed (defaults to true). -
setAutoCloseReader
Indicates if the file's stream is closed when the reader is closed (defaults to true). -
isLineageSupported
public boolean isLineageSupported()Description copied from class:PipelineInputIndicates if this input can save data lineage on the records it reads (false unless a subclass overrides it).- Overrides:
isLineageSupportedin classPipelineInput
-
generateJavaCode
Description copied from interface:JavaCodeGeneratorAppends Java source code representing this object to the given builder. -
generateJavaCodeImpl
Writes the reader declaration (and any format-specific setters), leaving the writer indented for the common setters. -
toRecord
Description copied from interface:RecordSerializableConverts this object's state to a record thatRecordSerializable.fromRecord(Record)can load.- Specified by:
toRecordin interfaceRecordSerializable- Overrides:
toRecordin classPipelineInput
-
fromRecord
Description copied from interface:RecordSerializableLoads this instance's state from a record and returnsthis(for fluid API call chaining). For fluid API call chaining, the overridden method should change the declared return type to its class.- Specified by:
fromRecordin interfaceRecordSerializable- Overrides:
fromRecordin classPipelineInput- Parameters:
source-- Returns:
- this instance.
-
toXmlElement
Description copied from interface:XmlSerializableReturns an element, created withdocument, describing this object; the default implementation throws aDataException.- Specified by:
toXmlElementin interfaceXmlSerializable- Overrides:
toXmlElementin classPipelineInput
-
fromXmlElement
- Specified by:
fromXmlElementin interfaceXmlSerializable- Overrides:
fromXmlElementin classPipelineInput
-