com.northconcepts.datapipeline.transform

Class Ngrams

java.lang.Object

com.northconcepts.datapipeline.transform.Transformer

com.northconcepts.datapipeline.transform.Ngrams

public class Ngrams
extends Transformer

Extracts each run/sequence of N words from the source field and stores them in a target field as an array. See n-gram, Bigrams, and Trigrams for more details.

Constructor Summary

Constructors

Constructor and Description
Ngrams(String sourceFieldPath, String targetFieldPath, int ngramCount)
Ngrams(String sourceFieldPath, String targetFieldPath, int ngramCount, Locale locale)

Method Summary

Modifier and Type	Method and Description
String	toString()
boolean	transform(Record record) Applies this transformer to a record on its way through a TransformingReader or TransformingWriter.

Methods inherited from class com.northconcepts.datapipeline.transform.Transformer

addExceptionProperties, close, geEndpoint, getEndpoint, getReader, getWriter, of, open, setWriter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Ngrams

public Ngrams(String sourceFieldPath,
              String targetFieldPath,
              int ngramCount,
              Locale locale)

Ngrams

public Ngrams(String sourceFieldPath,
              String targetFieldPath,
              int ngramCount)

Method Detail

transform

public boolean transform(Record record)
                  throws Throwable

Description copied from class: Transformer

Applies this transformer to a record on its way through a TransformingReader or TransformingWriter.

Contract for subclasses (see also docs/authoring/Transformer.md):

• Mutate record in place as needed.
• Return true to keep the record in the pipeline.
• Return false to drop the record (the host routes it to a discard writer if one is configured, otherwise it disappears).
• Throw to fail (or route to a discard writer, depending on host configuration).
• Do not call incrementRecordCount() on the host endpoint; the host handles it.

Specified by:

transform in class Transformer

Throws:

Throwable

toString

public String toString()

Overrides:

toString in class Object