Class Transformer

java.lang.Object
com.northconcepts.datapipeline.transform.Transformer
Direct Known Subclasses:
CopyField, FastRenameField, FieldTransformer, FlattenRecord, GetUrlQueryParam, LookupTransformer, MoveFieldAfter, MoveFieldBefore, MoveFieldToIndex, Ngrams, RemoveDuplicateFields, RemoveFields, RenameField, SchemaTransformer, SelectFields, SetBatchSequenceNumberField, SetCalculatedField, SetField, SetGroupSequenceNumberField, SetSequenceNumberField, SetUuidField, SplitArrayField, SplitWords

public abstract class Transformer extends Object
Base class for transformations that modify each record passing through a TransformingReader or TransformingWriter.
  • Constructor Details

    • Transformer

      public Transformer()
  • Method Details

    • getEndpoint

      public DataEndpoint getEndpoint()
      Returns the reader or writer this transformer was added to (the reader if both) or null if neither.
    • geEndpoint

      @Deprecated public DataEndpoint geEndpoint()
      Deprecated.
      Use getEndpoint() instead.
    • getReader

      public TransformingReader getReader()
      Returns the TransformingReader this transformer was added to or null if none.
    • getWriter

      public TransformingWriter getWriter()
      Returns the TransformingWriter this transformer was added to or null if none.
    • setWriter

      public void setWriter(TransformingWriter writer)
      Sets the host writer; called by TransformingWriter when this transformer is added to it.
    • addExceptionProperties

      public DataException addExceptionProperties(DataException exception)
      Adds this transformer's details to an exception raised by its host reader or writer; returns the same exception.
    • open

      protected void open()
      Called by TransformingReader or TransformingWriter when their open() method is called (before reading or writing starts).
    • close

      protected void close()
      Called by TransformingReader or TransformingWriter when their close() method is called (after reading or writing ends).
    • transform

      public abstract boolean transform(Record record) throws Throwable
      Applies this transformer to a record on its way through a TransformingReader or TransformingWriter.

      Contract for subclasses (see also docs/authoring/Transformer.md):

      • Mutate record in place as needed.
      • Return true to keep the record in the pipeline.
      • Return false to drop the record (the host routes it to a discard writer if one is configured, otherwise it disappears).
      • Throw to fail (or route to a discard writer, depending on host configuration).
      • Do not call incrementRecordCount() on the host endpoint; the host handles it.
      Throws:
      Throwable
    • of

      public static Transformer of(Function<Record,Boolean> function)
      Creates a Transformer from a Java Function. This allows creation of transformers using lambda expressions or method references.
      Parameters:
      function - the function that transforms records, returning true to continue or false to discard the record
      Returns:
      a Transformer that delegates to the function