Package com.northconcepts.datapipeline.transform

Class TransformingWriter

java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataWriter
com.northconcepts.datapipeline.core.ProxyWriter
com.northconcepts.datapipeline.transform.TransformingWriter
public class TransformingWriter extends ProxyWriter
A proxy that applies transformations to records passing through.

  • Constructor Details

    • TransformingWriter

      public TransformingWriter(DataWriter writer, DataWriter discardWriter, String discardReasonFieldName)
      Accepts the target DataWriter of the transform and the DataWriter to send discarded records to (due to exception).
      Parameters:
      writer - the target writer of the transform.
      discardWriter - the discard sink for bad records.
      discardReasonFieldName - the field in the record to place the description of the transformation that failed along with the exception message.

    • TransformingWriter

      public TransformingWriter(DataWriter writer, DataWriter discardWriter)
      Accepts the target DataWriter of the transform and the DataWriter to send discarded records to (due to exception).
      Parameters:
      writer - the target writer of the transform.
      discardWriter - the discard sink for bad records.

    • TransformingWriter

      public TransformingWriter(DataWriter writer)
      Accepts the target DataWriter of the transform.
      Parameters:
      writer - the target writer of the transform.

  • Method Details

    • getCondition

      public Filter getCondition()
      Returns the condition under which this transformation applies or null if it always applies.

    • setCondition

      public TransformingWriter setCondition(Filter condition)
      Sets the condition under which this transformation applies or null if it should always be applied.

    • isExceptionOnFailure

      public boolean isExceptionOnFailure()
      Indicates if a transformer that throws or returns false stops processing with an exception (default is true unless a discard writer is set).

    • setExceptionOnFailure

      public TransformingWriter setExceptionOnFailure(boolean exceptionOnFailure)
      Sets the flag to stop processing if an exception is thrown (default) or false if they should be ignored.

    • isRecordStackTraceInMessage

      public boolean isRecordStackTraceInMessage()
      Indicates if the warning added when a transformer returns false includes a stack trace (default is true).

    • setRecordStackTraceInMessage

      public TransformingWriter setRecordStackTraceInMessage(boolean recordStackTraceInMessage)
      Indicates if the warning added when a transformer returns false includes a stack trace (default is true).

    • add

      public TransformingWriter add(Transformer... transformer)
      Appends transformers; each record passes through them in the order added.

    • getCount

      public int getCount()
      Returns the number of transformers added.

    • get

      public Transformer get(int index)
      Returns the transformer at the given 0-based index.

    • getDiscardWriter

      public DataWriter getDiscardWriter()
      Returns the discard sink for bad records or null if one was not assigned.

    • getDiscardReasonFieldName

      public String getDiscardReasonFieldName()
      Returns the the field in the record to place the description of the transformation that failed along with the exception message.

    • open

      public void open() throws DataException
      Description copied from class: DataEndpoint
      Makes this endpoint ready for reading or writing.
      Overrides:
      open in class ProxyWriter
      Throws:
      DataException

    • close

      public void close() throws DataException
      Description copied from class: DataEndpoint
      Indicates that this endpoint has finished reading or writing.
      Overrides:
      close in class ProxyWriter
      Throws:
      DataException

    • interceptRecord

      protected Record interceptRecord(Record record) throws Throwable
      Description copied from class: ProxyWriter
      Overridden by subclasses to transform, drop, or observe a record on its way through this proxy.

      Contract for subclasses (see also docs/authoring/ProxyWriter.md):

      • Return the record (possibly modified) to forward it to the nested DataWriter.
      • Return null to drop the record; the framework will not write it downstream.
      • Throw to fail the pipeline; the framework wraps the throwable in a DataException with .setRecord(record).
      • The nested DataWriter is opened and closed for you; do not write to it here.
      • Do not call DataEndpoint.incrementRecordCount() here.
      Overrides:
      interceptRecord in class ProxyWriter
      Throws:
      Throwable

    • transformRecord

      protected boolean transformRecord(Record record, Transformer transformer, Messages messages)
      Applies the transformer to the record, passing any failure (an exception or a false result) to onFailure; returns true if the record should continue.

    • onFailure

      protected void onFailure(Record record, Transformer transformer, DataException exception, String message, Messages messages)
      Handles a failed transformation (exception is null if it returned false): sends the record to the discard writer if there is one, otherwise throws or, when exceptionOnFailure is off, adds warnings to messages.

    • addExceptionProperties

      public DataException addExceptionProperties(DataException exception)
      Description copied from class: Endpoint
      Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.
      Overrides:
      addExceptionProperties in class ProxyWriter