Package com.northconcepts.datapipeline.avro

Class AvroReader

java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataReader
com.northconcepts.datapipeline.internal.lang.IntegrationReader
com.northconcepts.datapipeline.avro.AvroReader
public class AvroReader extends IntegrationReader
Read an Apache Avro file and convert the contents into Records .

  • Constructor Details

    • AvroReader

      public AvroReader(File file)
      Read an Avro file.
      Parameters:
      file - the file to be read

    • AvroReader

      public AvroReader(InputStream inputStream)
      Read a stream of Avro data.
      Parameters:
      inputStream -

  • Method Details

    • open

      public void open() throws DataException
      Description copied from class: DataEndpoint
      Makes this endpoint ready for reading or writing.
      Overrides:
      open in class IntegrationReader
      Throws:
      DataException

    • close

      public void close() throws DataException
      Description copied from class: DataEndpoint
      Indicates that this endpoint has finished reading or writing.
      Overrides:
      close in class DataEndpoint
      Throws:
      DataException

    • readImpl

      protected Record readImpl() throws Throwable
      Description copied from class: DataReader
      Overridden by subclasses to read the next record from this DataReader. The default implementation of DataReader.read() now insures that this method will not be called again after it returns a null.

      If no record is available, null will be returned.

      Contract for subclasses (see also docs/authoring/DataReader.md):

      Specified by:
      readImpl in class DataReader
      Throws:
      Throwable

    • getMaxInvalidRecords

      public int getMaxInvalidRecords()
      Indicates the number of broken Avro records to skip before this reader should fail (default 1000).

    • setMaxInvalidRecords

      public AvroReader setMaxInvalidRecords(int maxInvalidRecords)
      Indicates the number of broken Avro records to skip before this reader should fail (default 1000).

    • getInvalidRecordCount

      public int getInvalidRecordCount()
      Returns the total number of broken Avro records this reader encountered.

    • getSchema

      public Schema getSchema()
      Returns the Avro schema of the data being read or null if the reader has not been opened.

    • addExceptionProperties

      public DataException addExceptionProperties(DataException exception)
      Description copied from class: Endpoint
      Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.
      Overrides:
      addExceptionProperties in class DataReader