Class AbstractReader
java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataReader
com.northconcepts.datapipeline.core.AbstractReader
- Direct Known Subclasses:
ExcelReader,GoogleSheetReader,ParsingReader,TextReader
Abstract super-class, with some common logic, for reading records. The only method subclasses must implement is
fillRecord(Record).-
Nested Class Summary
Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint
DataEndpoint.State -
Field Summary
FieldsFields inherited from class com.northconcepts.datapipeline.core.DataReader
fieldLineage, recordLineageFields inherited from class com.northconcepts.datapipeline.core.DataEndpoint
lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEYFields inherited from class com.northconcepts.datapipeline.core.Endpoint
BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZEFields inherited from class com.northconcepts.datapipeline.core.DataObject
id, log, name, TIMESTAMP_FORMAT -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionaddExceptionProperties(DataException exception) Adds this endpoint's current state to aDataException.protected abstract booleanfillRecord(Record record) Populates the given record with the next row's values; returns false at the end of the input.String[]Returns a copy of the field names that were set or read from the first row, or an empty array if there are none yet.intReturns the 0-based index of the last row to read or -1 to read to the end (default is -1).intReturns the 0-based index of the first row to read; earlier rows are skipped when the reader is opened (default is 0).booleanIndicates if the first row holds the field names instead of data (default is false).booleanIndicates that rows with only null values should not be returned by the reader (default is false).voidopen()Makes this endpoint ready for reading or writing.read()Reads the next record from thisDataReaderand increases the record-count by 1.protected RecordreadImpl()Overridden by subclasses to read the next record from thisDataReader.setFieldNames(String... fieldNames) Sets the names given to each record's fields, taking precedence over names in the first row; must be called beforeopen().setFieldNames(Collection<String> fieldNames) Sets the names given to each record's fields, taking precedence over names in the first row; must be called beforeopen().setFieldNamesInFirstRow(boolean fieldNamesInFirstRow) Indicates if the first row holds the field names instead of data (default is false); must be called beforeopen().setLastRow(int lastRow) Sets the 0-based index of the last row to read or -1 to read to the end (default is -1).setSkipEmptyRows(boolean skipEmptyRows) Indicates that rows with only null values should not be returned by the reader (default is false).setStartingRow(int startingRow) Sets the 0-based index of the first row to read; earlier rows are skipped when the reader is opened (default is 0).Methods inherited from class com.northconcepts.datapipeline.core.DataReader
addLineage, available, getBufferSize, getNestedEndpoint, getNestedReader, getReader, getRootEndpoint, getRootReader, isExhausted, isLineageSupported, isSaveLineage, peek, pop, push, setSaveLineage, skipMethods inherited from class com.northconcepts.datapipeline.core.DataEndpoint
close, decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toStringMethods inherited from class com.northconcepts.datapipeline.core.Endpoint
addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription
-
Field Details
-
fieldNames
-
currentRecord
-
startingRow
protected int startingRow -
lastRow
protected int lastRow
-
-
Constructor Details
-
AbstractReader
public AbstractReader()
-
-
Method Details
-
addExceptionProperties
Description copied from class:EndpointAdds this endpoint's current state to aDataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.- Overrides:
addExceptionPropertiesin classDataReader
-
isFieldNamesInFirstRow
public boolean isFieldNamesInFirstRow()Indicates if the first row holds the field names instead of data (default is false). -
setFieldNamesInFirstRow
Indicates if the first row holds the field names instead of data (default is false); must be called beforeopen(). -
getFieldNames
Returns a copy of the field names that were set or read from the first row, or an empty array if there are none yet. -
setFieldNames
Sets the names given to each record's fields, taking precedence over names in the first row; must be called beforeopen(). -
setFieldNames
Sets the names given to each record's fields, taking precedence over names in the first row; must be called beforeopen(). -
setStartingRow
Sets the 0-based index of the first row to read; earlier rows are skipped when the reader is opened (default is 0). -
getStartingRow
public int getStartingRow()Returns the 0-based index of the first row to read; earlier rows are skipped when the reader is opened (default is 0). -
setLastRow
Sets the 0-based index of the last row to read or -1 to read to the end (default is -1). -
getLastRow
public int getLastRow()Returns the 0-based index of the last row to read or -1 to read to the end (default is -1). -
setSkipEmptyRows
Indicates that rows with only null values should not be returned by the reader (default is false). -
isSkipEmptyRows
public boolean isSkipEmptyRows()Indicates that rows with only null values should not be returned by the reader (default is false). -
open
Description copied from class:DataEndpointMakes this endpoint ready for reading or writing.- Overrides:
openin classDataEndpoint- Throws:
DataException
-
read
Description copied from class:DataReaderReads the next record from thisDataReaderand increases the record-count by 1. This method will first read any pushed (DataReader.push(Record)) records before reading from the underlying source.If no record is available,
nullwill be returned. This method blocks until a record is available, the end of the stream is reached, or an exception is thrown.Any exception raised while reading will be converted to a
DataExceptionusingDataObject.exception(Throwable).Subclasses generally do not need to override this method, instead they should implement
DataReader.readImpl().- Overrides:
readin classDataReader- Throws:
DataException- See Also:
-
readImpl
Description copied from class:DataReaderOverridden by subclasses to read the next record from thisDataReader. The default implementation ofDataReader.read()now insures that this method will not be called again after it returns anull.If no record is available,
nullwill be returned.Contract for subclasses (see also
docs/authoring/DataReader.md):- Return
nullexactly once to signal end-of-stream. - Do not call this method directly —
DataReader.read()is the template method that wraps exceptions, increments the record count, and tracks lineage. - Do not call
DataEndpoint.incrementRecordCount()here;DataReader.read()already does. - Throw raw exceptions;
DataReader.read()wraps them viaexception(throwable).
- Specified by:
readImplin classDataReader- Throws:
Throwable
- Return
-
fillRecord
Populates the given record with the next row's values; returns false at the end of the input.- Throws:
Throwable
-