Package com.northconcepts.datapipeline.throttle

Class ThrottledWriter

java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataWriter
com.northconcepts.datapipeline.core.ProxyWriter
com.northconcepts.datapipeline.meter.MeteredWriter
com.northconcepts.datapipeline.throttle.ThrottledWriter
All Implemented Interfaces:
Metered, Throttled
public class ThrottledWriter extends MeteredWriter implements Throttled
A proxy that limits the rate (bytes/second) at which data is written.

  • Constructor Details

    • ThrottledWriter

      public ThrottledWriter(DataWriter writer, int bytesPerSecond)
      Creates a writer that limits writing to bytesPerSecond on average, counting each record's estimated size in bytes.

    • ThrottledWriter

      public ThrottledWriter(DataWriter writer, int unitsPerSecond, RecordMeter.MeterUnit units)
      Creates a writer that limits writing to unitsPerSecond on average, counting bytes or records depending on units.

  • Method Details

    • getThrottle

      public Throttle getThrottle()
      Specified by:
      getThrottle in interface Throttled

    • getMeasure

      public RecordMeter.MeterUnit getMeasure()
      Returns whether the limit counts records or their estimated size in bytes (defaults to BYTES).

    • setMeasure

      public ThrottledWriter setMeasure(RecordMeter.MeterUnit measure)
      Sets whether the limit counts records or their estimated size in bytes (defaults to BYTES).

    • interceptRecord

      protected Record interceptRecord(Record record) throws Throwable
      Description copied from class: ProxyWriter
      Overridden by subclasses to transform, drop, or observe a record on its way through this proxy.

      Contract for subclasses (see also docs/authoring/ProxyWriter.md):

      • Return the record (possibly modified) to forward it to the nested DataWriter.
      • Return null to drop the record; the framework will not write it downstream.
      • Throw to fail the pipeline; the framework wraps the throwable in a DataException with .setRecord(record).
      • The nested DataWriter is opened and closed for you; do not write to it here.
      • Do not call DataEndpoint.incrementRecordCount() here.
      Overrides:
      interceptRecord in class MeteredWriter
      Throws:
      Throwable

    • addExceptionProperties

      public DataException addExceptionProperties(DataException exception)
      Description copied from class: Endpoint
      Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.
      Overrides:
      addExceptionProperties in class MeteredWriter