Class JdbcUpsertWriter
java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataWriter
com.northconcepts.datapipeline.jdbc.JdbcUpsertWriter
Writes records to a database table using the upsert idiom (attempt to insert, but update if duplicate key exists).
The default strategy used is
GenericUpsert, however several other are provided: MergeUpsert, VariableFieldsUpsert, MySqlUpsert.-
Nested Class Summary
Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint
DataEndpoint.State -
Field Summary
FieldsFields inherited from class com.northconcepts.datapipeline.core.DataEndpoint
lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEYFields inherited from class com.northconcepts.datapipeline.core.Endpoint
BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZEFields inherited from class com.northconcepts.datapipeline.core.DataObject
id, log, name, TIMESTAMP_FORMAT -
Constructor Summary
ConstructorsConstructorDescriptionJdbcUpsertWriter(JdbcConnectionFactory factory, String tableName, IUpsert upsert, String... keyFieldNames) Creates a new instance with theautoCloseConnectionproperty set totrue, using a connection obtained from the factory immediately; at least one key field is required.JdbcUpsertWriter(JdbcConnectionFactory factory, String tableName, String... keyFieldNames) JdbcUpsertWriter(Connection connection, String tableName, IUpsert upsert, String... keyFieldNames) Creates a new instance with theautoCloseConnectionproperty set tofalse; at least one key field is required.JdbcUpsertWriter(Connection connection, String tableName, String... keyFieldNames) -
Method Summary
Modifier and TypeMethodDescriptionaddExceptionProperties(DataException exception) Adds this endpoint's current state to aDataException.voidclose()Indicates that this endpoint has finished reading or writing.booleanIndicates if the database connection in this endpoint should be closed when the endpoint is closed.intReturns the number of records to chunk together in batch updates; 1 or less (the default is -1) disables batching.getJdbcType(Class<?> type) TheTypesconstant (or a database specific value) assigned to the specified Java class type usingsetJdbcType(Class, int)or null if none was set.getJdbcType(String fieldName) TheTypesconstant (or a database specific value) assigned to the specified field usingsetJdbcType(String, int)or null if none was set.Returns the fields that identify the existing row to update.Returns the list of fields that must not be updated if the recored already exists (default null).booleanIndicates if updates are batched, i.e.booleanReturns true if this reader calls commit on the connection after every batch is sent.booleanisDebug()Indicates if the generated SQL should be logged (default false).voidopen()Makes this endpoint ready for reading or writing.setAutoCloseConnection(boolean closeConnection) Indicates if the database connection in this endpoint should be closed when the endpoint is closed.setBatchSize(int batchSize) Sets the number of records to chunk together in batch updates.setCommitBatch(boolean commitBatch) Indicates if this reader should call commit on the connection after every batch is sent, otherwise commit is called after the final batch update.setDebug(boolean debug) Indicates if the generated SQL should be logged (default false).setJdbcType(Class<?> type, int jdbcType) Forces field values of the specified Java class type to be sent to the database usingPreparedStatement.setObject(parameterIndex, fieldValue, jdbcType)(instead of the appropriatePreparedStatement.setXXX()method for its field's type).setJdbcType(String fieldName, int jdbcType) The given Java object will be converted to the given targetSqlType before being sent to the database.setNonUpdateFields(FieldList nonUpdateFields) Assigns the list of fields that must not be updated if the recored already exists (default null).setNonUpdateFields(String... nonUpdateFields) Assigns the list of fields that must not be updated if the record already exists (default null).voidsetParameterValue(Field field, int fieldIndex, PreparedStatement statement) Sets statement parameterfieldIndex + 1from the field, honoring any JDBC type assigned withsetJdbcType.voidsetParameterValue(Field field, int fieldIndex, PreparedStatement statement, int parameterIndex) Sets the 1-based statement parameter from the field, honoring any JDBC type assigned withsetJdbcType;fieldIndexis only used in error details.voidWrites the specified record to thisDataWriterand increases the record-count by 1.protected voidOverridden by subclasses to write the specified record to thisDataWriter.Methods inherited from class com.northconcepts.datapipeline.core.DataWriter
available, getNestedEndpoint, getNestedWriter, getRootEndpoint, getRootWriter, getWriterMethods inherited from class com.northconcepts.datapipeline.core.DataEndpoint
decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toStringMethods inherited from class com.northconcepts.datapipeline.core.Endpoint
addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription
-
Field Details
-
currentRecord
-
-
Constructor Details
-
JdbcUpsertWriter
public JdbcUpsertWriter(Connection connection, String tableName, IUpsert upsert, String... keyFieldNames) Creates a new instance with theautoCloseConnectionproperty set tofalse; at least one key field is required. -
JdbcUpsertWriter
-
JdbcUpsertWriter
public JdbcUpsertWriter(JdbcConnectionFactory factory, String tableName, IUpsert upsert, String... keyFieldNames) Creates a new instance with theautoCloseConnectionproperty set totrue, using a connection obtained from the factory immediately; at least one key field is required. -
JdbcUpsertWriter
-
-
Method Details
-
getConnection
-
getTableName
-
getAutoCloseConnection
public boolean getAutoCloseConnection()Indicates if the database connection in this endpoint should be closed when the endpoint is closed. The default isfalseif this instance was constructed with aConnectionortrueif constructed with aJdbcConnectionFactory. -
setAutoCloseConnection
Indicates if the database connection in this endpoint should be closed when the endpoint is closed. The default isfalseif this instance was constructed with aConnectionortrueif constructed with aJdbcConnectionFactory. -
getBatchSize
public int getBatchSize()Returns the number of records to chunk together in batch updates; 1 or less (the default is -1) disables batching. -
setBatchSize
Sets the number of records to chunk together in batch updates. Updates are not batched if this value is 1 or less. JdbcWriter defaults to non batching updates. -
isBatchMode
public boolean isBatchMode()Indicates if updates are batched, i.e. the batch size is greater than 1. -
isCommitBatch
public boolean isCommitBatch()Returns true if this reader calls commit on the connection after every batch is sent. -
setCommitBatch
Indicates if this reader should call commit on the connection after every batch is sent, otherwise commit is called after the final batch update. -
getKeyFields
Returns the fields that identify the existing row to update. -
getNonUpdateFields
Returns the list of fields that must not be updated if the recored already exists (default null). This method may returnnullor a list of empty fields. -
setNonUpdateFields
Assigns the list of fields that must not be updated if the recored already exists (default null). -
setNonUpdateFields
Assigns the list of fields that must not be updated if the record already exists (default null). -
getUpsert
-
setJdbcType
The given Java object will be converted to the given targetSqlType before being sent to the database.
Note that this method may be used to pass database-specific abstract data types.
- Parameters:
fieldName-jdbcType- the object containing the JDBC Type parameter value
-
getJdbcType
TheTypesconstant (or a database specific value) assigned to the specified field usingsetJdbcType(String, int)or null if none was set.- Returns:
- one of the
Typesconstants or a database specific value.
-
setJdbcType
Forces field values of the specified Java class type to be sent to the database usingPreparedStatement.setObject(parameterIndex, fieldValue, jdbcType)(instead of the appropriatePreparedStatement.setXXX()method for its field's type). This method may be used to pass database-specific abstract data types.- Parameters:
type- the Java class type to override.jdbcType- one of theTypesconstants or a database specific value.
-
getJdbcType
TheTypesconstant (or a database specific value) assigned to the specified Java class type usingsetJdbcType(Class, int)or null if none was set.- Returns:
- one of the
Typesconstants or a database specific value.
-
isDebug
public boolean isDebug()Indicates if the generated SQL should be logged (default false). -
setDebug
Indicates if the generated SQL should be logged (default false). -
addExceptionProperties
Description copied from class:EndpointAdds this endpoint's current state to aDataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.- Overrides:
addExceptionPropertiesin classDataWriter
-
setParameterValue
public void setParameterValue(Field field, int fieldIndex, PreparedStatement statement) throws SQLException Sets statement parameterfieldIndex + 1from the field, honoring any JDBC type assigned withsetJdbcType.- Throws:
SQLException
-
setParameterValue
public void setParameterValue(Field field, int fieldIndex, PreparedStatement statement, int parameterIndex) throws SQLException Sets the 1-based statement parameter from the field, honoring any JDBC type assigned withsetJdbcType;fieldIndexis only used in error details.- Throws:
SQLException
-
write
Description copied from class:DataWriterWrites the specified record to thisDataWriterand increases the record-count by 1. Any exception raised while writing will be converted to aDataExceptionusingDataObject.exception(Throwable).Subclasses generally do not need to override this method, instead they should implement
DataWriter.writeImpl(Record).- Overrides:
writein classDataWriter- Throws:
DataException
-
writeImpl
Description copied from class:DataWriterOverridden by subclasses to write the specified record to thisDataWriter.Contract for subclasses (see also
docs/authoring/DataWriter.md):- Do not call this method directly —
DataWriter.write(Record)is the template method that wraps exceptions, increments the record count, and attaches the offending record to thrownDataExceptions. - Do not call
DataEndpoint.incrementRecordCount()here;DataWriter.write(Record)already does. - Throw raw exceptions;
DataWriter.write(Record)wraps them viaexception(throwable).setRecord(record).
- Specified by:
writeImplin classDataWriter- Throws:
Throwable
- Do not call this method directly —
-
open
public void open()Description copied from class:DataEndpointMakes this endpoint ready for reading or writing.- Overrides:
openin classDataEndpoint
-
close
public void close()Description copied from class:DataEndpointIndicates that this endpoint has finished reading or writing.- Overrides:
closein classDataEndpoint
-