Package com.northconcepts.datapipeline.filter

Class FilteringReader

java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataReader
com.northconcepts.datapipeline.core.ProxyReader
com.northconcepts.datapipeline.filter.FilteringReader
Direct Known Subclasses:
ValidatingReader
public class FilteringReader extends ProxyReader
A proxy that chooses records using a filter criteria. Chosen records are returned by the read() methods, while rejected records disappear.

  • Constructor Details

    • FilteringReader

      public FilteringReader(DataReader reader, DataWriter discardWriter, String discardReasonFieldName)
      Accepts the DataReader to filter and the DataWriter to send discarded records to.
      Parameters:
      reader - the source reader to filter.
      discardWriter - the discard sink for disallowed records or null if they should simply disappear.
      discardReasonFieldName - the field in the record to place the description of the filter that disallowed it or null if one should not be added.

    • FilteringReader

      public FilteringReader(DataReader reader, DataWriter discardWriter)
      Accepts the DataReader to filter and the DataWriter to send discarded records to.
      Parameters:
      reader - the source reader to filter.
      discardWriter - the discard sink for disallowed records.

    • FilteringReader

      public FilteringReader(DataReader reader)
      Accepts the DataReader to filter.
      Parameters:
      reader - the source reader to filter.

  • Method Details

    • add

      public FilteringReader add(Filter... filters)
      Adds filters that every record must pass to be kept; they are applied in the order added.

    • getCurrentFilter

      public Filter getCurrentFilter()
      Returns the filter most recently used by this reader (such as the one that rejected the current record) or null if none was.

    • getDiscardWriter

      public DataWriter getDiscardWriter()
      Returns the discard sink for disallowed records or null if one was not assigned.

    • getDiscardReasonFieldName

      public String getDiscardReasonFieldName()
      Returns the field in the record to place the description of the filter that disallowed it or null if one should not be added.

    • getCount

      public int getCount()
      Returns the number of filters in this reader.

    • get

      public Filter get(int index)
      Returns the filter at the given 0-based index.

    • remove

      public FilteringReader remove(int index)
      Removes the filter at the given 0-based index.

    • removeAll

      public FilteringReader removeAll()

    • open

      public void open() throws DataException
      Description copied from class: DataEndpoint
      Makes this endpoint ready for reading or writing.
      Overrides:
      open in class ProxyReader
      Throws:
      DataException

    • close

      public void close() throws DataException
      Description copied from class: DataEndpoint
      Indicates that this endpoint has finished reading or writing.
      Overrides:
      close in class ProxyReader
      Throws:
      DataException

    • interceptRecord

      protected Record interceptRecord(Record record) throws Throwable
      Description copied from class: ProxyReader
      Overridden by subclasses to transform, drop, or observe a record on its way through this proxy.

      Contract for subclasses (see also docs/authoring/ProxyReader.md):

      • Return the record (possibly modified) to pass it downstream.
      • Return null to drop the record; the framework reads the next one.
      • Throw to fail the pipeline; the framework wraps the throwable in a DataException with .setRecord(record).
      • The nested DataReader is opened and closed for you; do not read from it here.
      • Do not call DataEndpoint.incrementRecordCount() here.
      Overrides:
      interceptRecord in class ProxyReader
      Throws:
      Throwable

    • discard

      protected Record discard(Record record, Filter filter)
      Handles a record rejected by filter; writes it to the discard writer (adding the discard reason field if configured) and returns null so it is dropped.

    • addExceptionProperties

      public DataException addExceptionProperties(DataException exception)
      Description copied from class: Endpoint
      Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.
      Overrides:
      addExceptionProperties in class ProxyReader