Class AvroPipelineOutput
java.lang.Object
com.northconcepts.datapipeline.foundations.core.Bean
com.northconcepts.datapipeline.foundations.core.FoundationObject
com.northconcepts.datapipeline.foundations.pipeline.PipelineObject
com.northconcepts.datapipeline.foundations.pipeline.PipelineOutput
com.northconcepts.datapipeline.foundations.pipeline.output.FileSinkPipelineOutput
com.northconcepts.datapipeline.avro.pipeline.output.AvroPipelineOutput
- All Implemented Interfaces:
DataExceptionContributor,JsonSerializable,RecordSerializable,XmlSerializable,DataWriterFactory,JavaCodeGenerator,Serializable
A pipeline output that writes Apache Avro data to a
FileSink using an AvroWriter.- See Also:
-
Field Summary
Fields inherited from class com.northconcepts.datapipeline.foundations.core.FoundationObject
internalId, internalName, log, TIMESTAMP_FORMATFields inherited from interface com.northconcepts.datapipeline.core.RecordSerializable
SERIALIZED_CLASS_NAME, TYPEFields inherited from interface com.northconcepts.datapipeline.core.XmlSerializable
XML_SERIALIZED_CLASS_NAME -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionfromRecord(Record source) Loads this instance's state from a record and returnsthis(for fluid API call chaining).fromXmlElement(Element pipelineInputElement) voidAppends Java source code representing this object to the given builder.intReturns the default scale used when writing BigDecimal values (default 5).intReturns the default precision used when writing BigDecimal invalid input: '&' BigInteger values (default 25).Returns the writer that receives records not matching the Avro schema or null if such records fail the write.getName()Returns a display name for this output, such as the destination's name followed by its format.booleanIndicates if characters other than letters, digits, '_' and '-' are removed from field names (default is true).booleanIndicates whether UUID values are written as strings or bytes.setDefaultBigDecimalScale(int defaultBigDecimalScale) Sets the default scale used when writing BigDecimal values (default 5).setDefaultBigNumberPrecision(int defaultBigNumberPrecision) Sets the default precision used when writing BigDecimal invalid input: '&' BigInteger values (default 25).setDiscardWriter(DataWriter discardWriter) Sets the writer that receives records not matching the Avro schema; when null, such records fail the write.setFileSink(FileSink fileSink) setRemoveUnsupportedChars(boolean removeUnsupportedChars) Indicates if characters other than letters, digits, '_' and '-' are removed from field names (default is true).setWriteUuidAsString(boolean writeUuidAsString) Indicates whether UUID values are written as strings or bytes.toRecord()Converts this object's state to a record thatRecordSerializable.fromRecord(Record)can load.toXmlElement(Document document) Returns an element, created withdocument, describing this object; the default implementation throws aDataException.Methods inherited from class com.northconcepts.datapipeline.foundations.pipeline.output.FileSinkPipelineOutput
generateSaveCodeMethods inherited from class com.northconcepts.datapipeline.foundations.pipeline.PipelineOutput
getNestedPipelineOutput, getPipelineOutput, getRootPipelineOutputMethods inherited from class com.northconcepts.datapipeline.foundations.core.FoundationObject
addExceptionProperties, assertValid, assertValid, clone, exception, exception, exception, getInternalId, getInternalName, resetInternalIdMethods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, waitMethods inherited from interface com.northconcepts.datapipeline.core.DataExceptionContributor
addExceptionProperties
-
Constructor Details
-
AvroPipelineOutput
public AvroPipelineOutput()
-
-
Method Details
-
createDataWriter
-
generateJavaCode
Description copied from interface:JavaCodeGeneratorAppends Java source code representing this object to the given builder. -
getFileSink
- Specified by:
getFileSinkin classFileSinkPipelineOutput
-
setFileSink
- Specified by:
setFileSinkin classFileSinkPipelineOutput
-
getDiscardWriter
Returns the writer that receives records not matching the Avro schema or null if such records fail the write. -
setDiscardWriter
Sets the writer that receives records not matching the Avro schema; when null, such records fail the write. -
isWriteUuidAsString
public boolean isWriteUuidAsString()Indicates whether UUID values are written as strings or bytes. If set to false, UUID values will be written as bytes.
Default value is true.
-
setWriteUuidAsString
Indicates whether UUID values are written as strings or bytes. If set to false, UUID values will be written as bytes.
Default value is true.
-
isRemoveUnsupportedChars
public boolean isRemoveUnsupportedChars()Indicates if characters other than letters, digits, '_' and '-' are removed from field names (default is true). -
setRemoveUnsupportedChars
Indicates if characters other than letters, digits, '_' and '-' are removed from field names (default is true). -
getDefaultBigDecimalScale
public int getDefaultBigDecimalScale()Returns the default scale used when writing BigDecimal values (default 5). -
setDefaultBigDecimalScale
Sets the default scale used when writing BigDecimal values (default 5). -
getDefaultBigNumberPrecision
public int getDefaultBigNumberPrecision()Returns the default precision used when writing BigDecimal invalid input: '&' BigInteger values (default 25). -
setDefaultBigNumberPrecision
Sets the default precision used when writing BigDecimal invalid input: '&' BigInteger values (default 25). -
getName
Description copied from class:PipelineOutputReturns a display name for this output, such as the destination's name followed by its format.- Overrides:
getNamein classFileSinkPipelineOutput
-
toRecord
Description copied from interface:RecordSerializableConverts this object's state to a record thatRecordSerializable.fromRecord(Record)can load.- Specified by:
toRecordin interfaceRecordSerializable- Overrides:
toRecordin classBean
-
fromRecord
Description copied from interface:RecordSerializableLoads this instance's state from a record and returnsthis(for fluid API call chaining). For fluid API call chaining, the overridden method should change the declared return type to its class.- Specified by:
fromRecordin interfaceRecordSerializable- Overrides:
fromRecordin classPipelineOutput- Parameters:
source-- Returns:
- this instance.
-
toXmlElement
Description copied from interface:XmlSerializableReturns an element, created withdocument, describing this object; the default implementation throws aDataException.- Specified by:
toXmlElementin interfaceXmlSerializable- Overrides:
toXmlElementin classPipelineOutput
-
fromXmlElement
-