Package com.northconcepts.datapipeline.jdbc

Class JdbcMultiWriter

java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataWriter
com.northconcepts.datapipeline.core.ProxyWriter
com.northconcepts.datapipeline.jdbc.JdbcMultiWriter
public class JdbcMultiWriter extends ProxyWriter
Writes records to a database table using 1 or more connections; each connection writing asynchronously using a separate thread.

  • Constructor Details

    • JdbcMultiWriter

      public JdbcMultiWriter(DataSource dataSource, int connections, int maxQueuedRecordsPerConnection, String tableName, IInsert insert)
      Creates a writer that obtains its connections from the data source when opened.

    • JdbcMultiWriter

      public JdbcMultiWriter(DataSource dataSource, int connections, int maxQueuedRecordsPerConnection, String tableName)
      Creates a writer using PreparedStatementInsert that obtains its connections from the data source when opened.

    • JdbcMultiWriter

      public JdbcMultiWriter(JdbcConnectionFactory factory, int connections, int maxQueuedRecordsPerConnection, String tableName, IInsert insert) throws DataException
      Creates a writer that obtains its connections from the factory when opened.
      Throws:
      DataException

    • JdbcMultiWriter

      public JdbcMultiWriter(JdbcConnectionFactory factory, int connections, int maxQueuedRecordsPerConnection, String tableName) throws DataException
      Creates a writer using PreparedStatementInsert that obtains its connections from the factory when opened.
      Throws:
      DataException

  • Method Details

    • getConnections

      public int getConnections()
      Returns the number of connections, each written to by its own thread.

    • getJdbcWriter

      public JdbcWriter getJdbcWriter(int index)
      Returns the writer for the connection at the 0-based index or null before open().

    • getAsyncWriter

      public AsyncWriter getAsyncWriter(int index)
      Returns the asynchronous writer feeding the connection at the 0-based index or null before open().

    • getBatchSize

      public int getBatchSize()
      Returns the number of records to chunk together in batch updates; 1 or less (the default is -1) disables batching.

    • setBatchSize

      public JdbcMultiWriter setBatchSize(int batchSize)
      Sets the number of records to chunk together in batch updates. Updates are not batched if this value is 1 or less. JdbcWriter defaults to non batching updates.

    • isCommitBatch

      public boolean isCommitBatch()
      Indicates if this writer should call commit on each connection after every batch is sent, otherwise, commit is called after the final batch update (default is false).

    • setCommitBatch

      public JdbcMultiWriter setCommitBatch(boolean commitBatch)
      Indicates if this writer should call commit on each connection after every batch is sent, otherwise, commit is called after the final batch update (default is false).

    • getInsert

      public IInsert getInsert()
      Returns the insert strategy; each connection's writer uses its own clone.

    • setJdbcType

      public JdbcMultiWriter setJdbcType(String fieldName, int jdbcType)

      The given Java object will be converted to the given targetSqlType before being sent to the database.

      Note that this method may be used to pass database-specific abstract data types.

      Parameters:
      fieldName -
      jdbcType - the object containing the JDBC Type parameter value

    • setJdbcType

      public JdbcMultiWriter setJdbcType(Class<?> type, int jdbcType)
      Forces field values of the specified Java class type to be sent to the database using PreparedStatement.setObject(parameterIndex, fieldValue, jdbcType) (instead of the appropriate PreparedStatement.setXXX() method for its field's type). This method may be used to pass database-specific abstract data types.
      Parameters:
      type - the Java class type to override.
      jdbcType - one of the Types constants or a database specific value.

    • getJdbcType

      public Integer getJdbcType(Class<?> type)
      The Types constant (or a database specific value) assigned to the specified Java class type using setJdbcType(Class, int) or null if none was set.
      Returns:
      one of the Types constants or a database specific value.

    • isDebug

      public boolean isDebug()
      Indicates if the generated SQL should be logged (default false).

    • setDebug

      public JdbcMultiWriter setDebug(boolean debug)
      Indicates if the generated SQL should be logged (default false).

    • open

      public void open() throws DataException
      Description copied from class: DataEndpoint
      Makes this endpoint ready for reading or writing.
      Overrides:
      open in class ProxyWriter
      Throws:
      DataException

    • createJdbcWriter

      protected JdbcWriter createJdbcWriter(Connection connection, String tableName, IInsert insertStrategy)
      Creates the writer for a connection obtained in open(); the default copies this writer's settings and makes the new writer close the connection.

    • writeImpl

      protected void writeImpl(Record record) throws Throwable
      Description copied from class: DataWriter
      Overridden by subclasses to write the specified record to this DataWriter.

      Contract for subclasses (see also docs/authoring/DataWriter.md):

      Overrides:
      writeImpl in class ProxyWriter
      Throws:
      Throwable

    • getException

      public DataException getException()
      Returns an exception thrown by one of the internal AsyncWriters if any.

    • getExceptions

      public List<DataException> getExceptions()
      Returns all exceptions thrown by the internal AsyncWriters (the list will not be null, but may contain no elements).

    • rethrowAsyncException

      public void rethrowAsyncException() throws DataException
      Re-throws an exception thrown by one of the internal AsyncWriters or returns silently if no exceptions were thrown.
      Throws:
      DataException

    • addExceptionProperties

      public DataException addExceptionProperties(DataException exception)
      Description copied from class: Endpoint
      Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.
      Overrides:
      addExceptionProperties in class ProxyWriter