Class JdbcReader
java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataReader
com.northconcepts.datapipeline.jdbc.JdbcReader
Obtains records from a database query.
-
Nested Class Summary
Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint
DataEndpoint.State -
Field Summary
FieldsFields inherited from class com.northconcepts.datapipeline.core.DataReader
fieldLineage, recordLineageFields inherited from class com.northconcepts.datapipeline.core.DataEndpoint
lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEYFields inherited from class com.northconcepts.datapipeline.core.Endpoint
BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZEFields inherited from class com.northconcepts.datapipeline.core.DataObject
id, log, name, TIMESTAMP_FORMAT -
Constructor Summary
ConstructorsConstructorDescriptionJdbcReader(JdbcConnectionFactory factory, String queryString) Creates a new instance with theautoCloseConnectionproperty set totrue.JdbcReader(JdbcConnectionFactory factory, String queryString, Object... parameters) Creates a new instance with theautoCloseConnectionproperty set totrue.JdbcReader(JdbcConnectionFactory factory, String queryString, Object[] parameters, int[] parameterSqlTypes) Creates a new instance with theautoCloseConnectionproperty set totrue.JdbcReader(Connection connection, String queryString) Creates a new instance with theautoCloseConnectionproperty set tofalse.JdbcReader(Connection connection, String queryString, Object... parameters) Creates a new instance with theautoCloseConnectionproperty set tofalse.JdbcReader(Connection connection, String queryString, Object[] parameters, int[] parameterSqlTypes) Creates a new instance with theautoCloseConnectionproperty set tofalse. -
Method Summary
Modifier and TypeMethodDescriptionaddExceptionProperties(DataException exception) Adds this endpoint's current state to aDataException.protected RecordaddLineage(Record record) Called byDataReader.read()for each record fromDataReader.readImpl()while lineage is saved; the default copiesrecordLineageinto every field along with its original index and name.voidclose()Indicates that this endpoint has finished reading or writing.booleanIndicates if the database connection in this endpoint should be closed when the endpoint is closed.The fetch size to set on theStatementusingStatement.setFetchSize(int)ornullif the connection's default should be used (defaultnull).booleanIndicates if column labels (or aliases) are used for field names instead of the underlying, real column names (defaults tofalse).Returns the strategy that maps column types and reads values (defaults toJdbcValueReader.OPINIONATED).booleanIndicates if this reader can capture record and field lineage (false unless overridden by a reader that supports it).booleanIndicates if database types should be mapped to java type based only onjava.sql.Typesas defined by Sun/Oracle (defaults to false).voidopen()Makes this endpoint ready for reading or writing.read()Reads the next record from thisDataReaderand increases the record-count by 1.protected RecordreadImpl()Overridden by subclasses to read the next record from thisDataReader.setAutoCloseConnection(boolean closeConnection) Indicates if the database connection in this endpoint should be closed when the endpoint is closed.setFetchSize(Integer fetchSize) The fetch size to set on theStatementusingStatement.setFetchSize(int)ornullif the connection's default should be used (defaultnull).setSaveLineage(boolean saveLineage) Indicates if record and field lineage is captured for each record read (default is false); enabling it throws ifDataReader.isLineageSupported()is false or the product edition does not include lineage.setStrictTypeMapping(boolean strictTypeMapping) Indicates if database types should be mapped to java type based only onjava.sql.Typesas defined by Sun/Oracle (defaults to false).setUseColumnLabel(boolean useColumnLabel) When set totrue, column labels (or aliases) will be used for field names instead of the underlying, real column names.setValueReader(JdbcValueReader valueReader) Sets the strategy that maps column types and reads values;nullrestoresJdbcValueReader.OPINIONATED.Methods inherited from class com.northconcepts.datapipeline.core.DataReader
available, getBufferSize, getNestedEndpoint, getNestedReader, getReader, getRootEndpoint, getRootReader, isExhausted, isSaveLineage, peek, pop, push, skipMethods inherited from class com.northconcepts.datapipeline.core.DataEndpoint
decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toStringMethods inherited from class com.northconcepts.datapipeline.core.Endpoint
addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription
-
Field Details
-
currentRecord
-
-
Constructor Details
-
JdbcReader
Creates a new instance with theautoCloseConnectionproperty set tofalse.- Parameters:
connection-queryString-
-
JdbcReader
Creates a new instance with theautoCloseConnectionproperty set tofalse.- Parameters:
connection-queryString-parameters-
-
JdbcReader
public JdbcReader(Connection connection, String queryString, Object[] parameters, int[] parameterSqlTypes) Creates a new instance with theautoCloseConnectionproperty set tofalse.- Parameters:
connection-queryString-parameters-parameterSqlTypes-
-
JdbcReader
Creates a new instance with theautoCloseConnectionproperty set totrue.- Parameters:
factory-queryString-- Throws:
DataException
-
JdbcReader
Creates a new instance with theautoCloseConnectionproperty set totrue.- Parameters:
factory-queryString-parameters-
-
JdbcReader
public JdbcReader(JdbcConnectionFactory factory, String queryString, Object[] parameters, int[] parameterSqlTypes) Creates a new instance with theautoCloseConnectionproperty set totrue.- Parameters:
factory-queryString-parameters-parameterSqlTypes-
-
-
Method Details
-
getAutoCloseConnection
public boolean getAutoCloseConnection()Indicates if the database connection in this endpoint should be closed when the endpoint is closed. The default isfalseif this instance was constructed with aConnectionortrueif constructed with aJdbcConnectionFactory. -
setAutoCloseConnection
Indicates if the database connection in this endpoint should be closed when the endpoint is closed. The default isfalseif this instance was constructed with aConnectionortrueif constructed with aJdbcConnectionFactory. -
getValueReader
Returns the strategy that maps column types and reads values (defaults toJdbcValueReader.OPINIONATED). -
setValueReader
Sets the strategy that maps column types and reads values;nullrestoresJdbcValueReader.OPINIONATED. -
getUseColumnLabel
public boolean getUseColumnLabel()Indicates if column labels (or aliases) are used for field names instead of the underlying, real column names (defaults tofalse). -
getFetchSize
The fetch size to set on theStatementusingStatement.setFetchSize(int)ornullif the connection's default should be used (defaultnull). -
setFetchSize
The fetch size to set on theStatementusingStatement.setFetchSize(int)ornullif the connection's default should be used (defaultnull). -
setUseColumnLabel
When set totrue, column labels (or aliases) will be used for field names instead of the underlying, real column names. Defaults tofalse. -
setSaveLineage
Description copied from class:DataReaderIndicates if record and field lineage is captured for each record read (default is false); enabling it throws ifDataReader.isLineageSupported()is false or the product edition does not include lineage.- Overrides:
setSaveLineagein classDataReader
-
isStrictTypeMapping
public boolean isStrictTypeMapping()Indicates if database types should be mapped to java type based only onjava.sql.Typesas defined by Sun/Oracle (defaults to false). Otherwise, the column display size and scale will be considered for numeric types.- See Also:
-
setStrictTypeMapping
Indicates if database types should be mapped to java type based only onjava.sql.Typesas defined by Sun/Oracle (defaults to false). Otherwise, the column display size and scale will be considered for numeric types.- See Also:
-
addExceptionProperties
Description copied from class:EndpointAdds this endpoint's current state to aDataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.- Overrides:
addExceptionPropertiesin classDataReader
-
read
Description copied from class:DataReaderReads the next record from thisDataReaderand increases the record-count by 1. This method will first read any pushed (DataReader.push(Record)) records before reading from the underlying source.If no record is available,
nullwill be returned. This method blocks until a record is available, the end of the stream is reached, or an exception is thrown.Any exception raised while reading will be converted to a
DataExceptionusingDataObject.exception(Throwable).Subclasses generally do not need to override this method, instead they should implement
DataReader.readImpl().- Overrides:
readin classDataReader- Throws:
DataException- See Also:
-
readImpl
Description copied from class:DataReaderOverridden by subclasses to read the next record from thisDataReader. The default implementation ofDataReader.read()now insures that this method will not be called again after it returns anull.If no record is available,
nullwill be returned.Contract for subclasses (see also
docs/authoring/DataReader.md):- Return
nullexactly once to signal end-of-stream. - Do not call this method directly —
DataReader.read()is the template method that wraps exceptions, increments the record count, and tracks lineage. - Do not call
DataEndpoint.incrementRecordCount()here;DataReader.read()already does. - Throw raw exceptions;
DataReader.read()wraps them viaexception(throwable).
- Specified by:
readImplin classDataReader- Throws:
Throwable
- Return
-
isLineageSupported
public boolean isLineageSupported()Description copied from class:DataReaderIndicates if this reader can capture record and field lineage (false unless overridden by a reader that supports it).- Overrides:
isLineageSupportedin classDataReader
-
addLineage
Description copied from class:DataReaderCalled byDataReader.read()for each record fromDataReader.readImpl()while lineage is saved; the default copiesrecordLineageinto every field along with its original index and name. Overrides set their source details onrecordLineagefirst and end withsuper.addLineage(record).- Overrides:
addLineagein classDataReader
-
open
public void open()Description copied from class:DataEndpointMakes this endpoint ready for reading or writing.- Overrides:
openin classDataEndpoint
-
close
public void close()Description copied from class:DataEndpointIndicates that this endpoint has finished reading or writing.- Overrides:
closein classDataEndpoint
-