Class OrcPipelineInput

Method Details

• getName

  public String getName()

  Description copied from class: PipelineInput

  Returns a display name for this input, such as the source's name followed by its format.

  Overrides:

  getName in class FileSourcePipelineInput

• getFileSource

  public FileSource getFileSource()

  Specified by:

  getFileSource in class FileSourcePipelineInput

• setFileSource

  public OrcPipelineInput setFileSource(FileSource fileSource)

  Specified by:

  setFileSource in class FileSourcePipelineInput

• isSaveLineage

  public boolean isSaveLineage()

  Description copied from class: PipelineInput

  Indicates if data lineage (where each record and field came from) is saved on the records read (defaults to false).

  Overrides:

  isSaveLineage in class PipelineInput

• setSaveLineage

  public OrcPipelineInput setSaveLineage(boolean saveLineage)

  Description copied from class: PipelineInput

  Indicates if data lineage (where each record and field came from) is saved on the records read (defaults to false); enabling it throws an exception if lineage is not supported.

  Overrides:

  setSaveLineage in class PipelineInput

• createDataReader

  public DataReader createDataReader()

• generateJavaCode

  public void generateJavaCode(JavaCodeBuilder javaCodeBuilder)

  Description copied from interface: JavaCodeGenerator

  Appends Java source code representing this object to the given builder.

• isLineageSupported

  public boolean isLineageSupported()

  Description copied from class: PipelineInput

  Indicates if this input can save data lineage on the records it reads (false unless a subclass overrides it).

  Overrides:

  isLineageSupported in class PipelineInput

• getBatchSize

  public int getBatchSize()

  Indicates the maximum number of records to buffer when reading ORC data (default 1024).

• setBatchSize

  public OrcPipelineInput setBatchSize(int batchSize)

  Indicates the maximum number of records to buffer when reading ORC data (default 1024).

• getSchema

  public TypeDescription getSchema()

  Indicates the schema used to read the file.

• setSchema

  public OrcPipelineInput setSchema(String schema)

  Sets the schema used to read the file as a string.

• setSchema

  public OrcPipelineInput setSchema(TypeDescription schema)

  Indicates the schema used to read the file.

• getColumns

  public FieldList getColumns()

  Indicates the columns to read from ORC file.

• setColumns

  public OrcPipelineInput setColumns(FieldList columns)

  Indicates the columns to read from ORC file.

• getConfig

  public Map<String,String> getConfig()

  Returns the Orc configuration parameters.

• addConfig

  public OrcPipelineInput addConfig(Map<String,String> config)

  Sets the Orc configuration parameters.

• addConfig

  public OrcPipelineInput addConfig(String key, String value)

  Adds a parameter to the Hadoop

  invalid reference

  Configuration

  used to read the file.

• toRecord

  public Record toRecord()

  Description copied from interface: RecordSerializable

  Converts this object's state to a record that RecordSerializable.fromRecord(Record) can load.

  Specified by:

  toRecord in interface RecordSerializable

  Overrides:

  toRecord in class PipelineInput

• fromRecord

  public OrcPipelineInput fromRecord(Record source)

  Description copied from interface: RecordSerializable

  Loads this instance's state from a record and returns this (for fluid API call chaining). For fluid API call chaining, the overridden method should change the declared return type to its class.

  Specified by:

  fromRecord in interface RecordSerializable

  Overrides:

  fromRecord in class PipelineInput

  Parameters:

  source -

  Returns:

  this instance.

• toXmlElement

  public Element toXmlElement(Document document)

  Description copied from interface: XmlSerializable

  Returns an element, created with document, describing this object; the default implementation throws a DataException.

  Specified by:

  toXmlElement in interface XmlSerializable

  Overrides:

  toXmlElement in class PipelineInput

• fromXmlElement

  public OrcPipelineInput fromXmlElement(Element pipelineInputElement)

  Specified by:

  fromXmlElement in interface XmlSerializable

  Overrides:

  fromXmlElement in class PipelineInput