Class AvroWriter
Write
Records to an Apache Avro file.-
Nested Class Summary
Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint
DataEndpoint.State -
Field Summary
Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint
lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEYFields inherited from class com.northconcepts.datapipeline.core.Endpoint
BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZEFields inherited from class com.northconcepts.datapipeline.core.DataObject
id, log, name, TIMESTAMP_FORMAT -
Constructor Summary
ConstructorsConstructorDescriptionAvroWriter(File file, DataWriter discardWriter) Write Avro data to the specified file.AvroWriter(OutputStream outputStream, DataWriter discardWriter) Write Avro data to anOutputStream. -
Method Summary
Modifier and TypeMethodDescriptionaddExceptionProperties(DataException exception) Adds this endpoint's current state to aDataException.voidclose()Indicates that this endpoint has finished reading or writing.intReturns the default scale used when writing BigDecimal values (default 5).intReturns the default precision used when writing BigDecimal invalid input: '&' BigInteger values (default 25).longReturns the number of discardedRecordsbooleanReturns whether unsupported characters are removed from field names.booleanIndicates whether UUID values are written as strings or bytes.voidopen()Makes this endpoint ready for reading or writing.setDefaultBigDecimalScale(int defaultBigDecimalScale) Sets the default scale used when writing BigDecimal values (default 5).setDefaultBigNumberPrecision(int defaultBigNumberPrecision) Sets the default precision used when writing BigDecimal invalid input: '&' BigInteger values (default 25).setRemoveUnsupportedChars(boolean removeUnsupportedChars) Sets whether unsupported characters are removed from field names.setWriteUuidAsString(boolean writeUuidAsString) Indicates whether UUID values are written as strings or bytes.protected voidOverridden by subclasses to write the specified record to thisDataWriter.Methods inherited from class com.northconcepts.datapipeline.core.DataWriter
available, getNestedEndpoint, getNestedWriter, getRootEndpoint, getRootWriter, getWriter, writeMethods inherited from class com.northconcepts.datapipeline.core.DataEndpoint
decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toStringMethods inherited from class com.northconcepts.datapipeline.core.Endpoint
addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription
-
Constructor Details
-
AvroWriter
Write Avro data to the specified file.- Parameters:
file- the file to write todiscardWriter- discard sink for records not conforming to the Avro schema
-
AvroWriter
Write Avro data to anOutputStream.- Parameters:
outputStream- the target streamdiscardWriter- discard sink for records not conforming to the Avro schema
-
-
Method Details
-
getDiscardedRecordCount
public long getDiscardedRecordCount()Returns the number of discardedRecords- Returns:
- long number of discarded records
-
isWriteUuidAsString
public boolean isWriteUuidAsString()Indicates whether UUID values are written as strings or bytes. If set to false, UUID values will be written as bytes.
Default value is true.
-
setWriteUuidAsString
Indicates whether UUID values are written as strings or bytes. If set to false, UUID values will be written as bytes.
Default value is true.
-
getDefaultBigDecimalScale
public int getDefaultBigDecimalScale()Returns the default scale used when writing BigDecimal values (default 5). -
setDefaultBigDecimalScale
Sets the default scale used when writing BigDecimal values (default 5). -
getDefaultBigNumberPrecision
public int getDefaultBigNumberPrecision()Returns the default precision used when writing BigDecimal invalid input: '&' BigInteger values (default 25). -
setDefaultBigNumberPrecision
Sets the default precision used when writing BigDecimal invalid input: '&' BigInteger values (default 25). -
isRemoveUnsupportedChars
public boolean isRemoveUnsupportedChars()Returns whether unsupported characters are removed from field names. -
setRemoveUnsupportedChars
Sets whether unsupported characters are removed from field names. -
open
Description copied from class:DataEndpointMakes this endpoint ready for reading or writing.- Overrides:
openin classIntegrationWriter- Throws:
DataException
-
close
Description copied from class:DataEndpointIndicates that this endpoint has finished reading or writing.- Overrides:
closein classDataEndpoint- Throws:
DataException
-
writeImpl
Description copied from class:DataWriterOverridden by subclasses to write the specified record to thisDataWriter.Contract for subclasses (see also
docs/authoring/DataWriter.md):- Do not call this method directly —
DataWriter.write(Record)is the template method that wraps exceptions, increments the record count, and attaches the offending record to thrownDataExceptions. - Do not call
DataEndpoint.incrementRecordCount()here;DataWriter.write(Record)already does. - Throw raw exceptions;
DataWriter.write(Record)wraps them viaexception(throwable).setRecord(record).
- Specified by:
writeImplin classDataWriter- Throws:
Throwable
- Do not call this method directly —
-
addExceptionProperties
Description copied from class:EndpointAdds this endpoint's current state to aDataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.- Overrides:
addExceptionPropertiesin classDataWriter
-