com.northconcepts.datapipeline.transform

Class Transformer

java.lang.Object

com.northconcepts.datapipeline.transform.Transformer

Direct Known Subclasses:

CopyField, FastRenameField, FieldTransformer, FlattenRecord, GetUrlQueryParam, LookupTransformer, MoveFieldAfter, MoveFieldBefore, MoveFieldToIndex, Ngrams, RemoveDuplicateFields, RemoveFields, RenameField, SchemaTransformer, SelectFields, SetBatchSequenceNumberField, SetCalculatedField, SetField, SetGroupSequenceNumberField, SetSequenceNumberField, SetUuidField, SplitArrayField, SplitWords

public abstract class Transformer
extends Object

Constructor Summary

Constructors

Constructor and Description
Transformer()

Method Summary

Modifier and Type	Method and Description
DataException	addExceptionProperties(DataException exception)
protected void	close() Called by TransformingReader or TransformingWriter when their close() method is called (after reading or writing ends).
DataEndpoint	geEndpoint() Deprecated. Use getEndpoint() instead.
DataEndpoint	getEndpoint()
TransformingReader	getReader()
TransformingWriter	getWriter()
static Transformer	of(Function<Record,Boolean> function) Creates a Transformer from a Java Function.
protected void	open() Called by TransformingReader or TransformingWriter when their open() method is called (before reading or writing starts).
void	setWriter(TransformingWriter writer)
abstract boolean	transform(Record record) Applies this transformer to a record on its way through a TransformingReader or TransformingWriter.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Transformer

public Transformer()

Method Detail

getEndpoint

public DataEndpoint getEndpoint()

geEndpoint

@Deprecated
public DataEndpoint geEndpoint()

Deprecated. Use getEndpoint() instead.

getReader

public TransformingReader getReader()

getWriter

public TransformingWriter getWriter()

setWriter

public void setWriter(TransformingWriter writer)

addExceptionProperties

public DataException addExceptionProperties(DataException exception)

open

protected void open()

Called by TransformingReader or TransformingWriter when their open() method is called (before reading or writing starts).

close

protected void close()

Called by TransformingReader or TransformingWriter when their close() method is called (after reading or writing ends).

transform

public abstract boolean transform(Record record)
                           throws Throwable

Applies this transformer to a record on its way through a TransformingReader or TransformingWriter.

Contract for subclasses (see also docs/authoring/Transformer.md):

• Mutate record in place as needed.
• Return true to keep the record in the pipeline.
• Return false to drop the record (the host routes it to a discard writer if one is configured, otherwise it disappears).
• Throw to fail (or route to a discard writer, depending on host configuration).
• Do not call incrementRecordCount() on the host endpoint; the host handles it.

Throws:

Throwable

of

public static Transformer of(Function<Record,Boolean> function)

Creates a Transformer from a Java Function. This allows creation of transformers using lambda expressions or method references.

Parameters:

function - the function that transforms records, returning true to continue or false to discard the record

Returns:

a Transformer that delegates to the function