Package com.northconcepts.datapipeline.foundations.pipeline

Class PipelineInput

java.lang.Object
com.northconcepts.datapipeline.foundations.core.Bean
com.northconcepts.datapipeline.foundations.core.FoundationObject
com.northconcepts.datapipeline.foundations.pipeline.PipelineObject
com.northconcepts.datapipeline.foundations.pipeline.PipelineInput
All Implemented Interfaces:
DataExceptionContributor, JsonSerializable, RecordSerializable, XmlSerializable, DataReaderFactory, JavaCodeGenerator, Serializable
Direct Known Subclasses:
DataReaderPipelineInput, DatasetPipelineInput, FileSourcePipelineInput, JdbcPipelineInput, ProxyPipelineInput
public abstract class PipelineInput extends PipelineObject implements DataReaderFactory
Base class for the sources a pipeline reads its records from, such as CSV, Excel, fixed-width, JSON or XML files, JDBC queries and datasets.
See Also:

  • Constructor Details

    • PipelineInput

      public PipelineInput()

  • Method Details

    • getName

      public abstract String getName()
      Returns a display name for this input, such as the source's name followed by its format.

    • isLineageSupported

      public boolean isLineageSupported()
      Indicates if this input can save data lineage on the records it reads (false unless a subclass overrides it).

    • isSaveLineage

      public boolean isSaveLineage()
      Indicates if data lineage (where each record and field came from) is saved on the records read (defaults to false).

    • getNestedPipelineInput

      public PipelineInput getNestedPipelineInput()
      Returns the PipelineInput held inside this one or null if there isn't one.

    • getRootPipelineInput

      public PipelineInput getRootPipelineInput()
      Returns the deepest, nested PipelineInput held inside this one, otherwise this instance is returned if there aren't any nested PipelineInputs.

    • getPipelineInput

      public <T extends PipelineInput> T getPipelineInput(Class<T> type)
      Searches for a nested PipelineInput of the specified type within this pipeline input hierarchy.

      This method first checks if the current instance is assignable to the specified type. If not, it traverses through all nested PipelineInputs to find one that matches the type.

      Type Parameters:
      T - the type of PipelineInput to search for
      Parameters:
      type - the class of the PipelineInput type to find
      Returns:
      the first matching PipelineInput of the specified type, or null if not found

    • setSaveLineage

      public PipelineInput setSaveLineage(boolean saveLineage)
      Indicates if data lineage (where each record and field came from) is saved on the records read (defaults to false); enabling it throws an exception if lineage is not supported.

    • toRecord

      public Record toRecord()
      Description copied from interface: RecordSerializable
      Converts this object's state to a record that RecordSerializable.fromRecord(Record) can load.
      Specified by:
      toRecord in interface RecordSerializable
      Overrides:
      toRecord in class Bean

    • fromRecord

      public PipelineInput fromRecord(Record source)
      Description copied from interface: RecordSerializable
      Loads this instance's state from a record and returns this (for fluid API call chaining). For fluid API call chaining, the overridden method should change the declared return type to its class.
      Specified by:
      fromRecord in interface RecordSerializable
      Overrides:
      fromRecord in class Bean
      Parameters:
      source -
      Returns:
      this instance.

    • toXmlElement

      public Element toXmlElement(Document document)
      Description copied from interface: XmlSerializable
      Returns an element, created with document, describing this object; the default implementation throws a DataException.
      Specified by:
      toXmlElement in interface XmlSerializable

    • fromXmlElement

      public XmlSerializable fromXmlElement(Element element)
      Specified by:
      fromXmlElement in interface XmlSerializable