Class BufferStrategy

java.lang.Object
com.northconcepts.datapipeline.buffer.BufferStrategy

public abstract class BufferStrategy extends Object
Determines when BufferedReader closes an open sliding buffer.
  • Field Details

    • CLOSE_BUFFER_MARGIN_MILLIS

      public static final long CLOSE_BUFFER_MARGIN_MILLIS
      See Also:
  • Constructor Details

    • BufferStrategy

      public BufferStrategy()
  • Method Details

    • shouldClose

      public abstract boolean shouldClose(RecordBuffer buffer, BufferedReader bufferedReader, Record record)
      Indicates if the open buffer should be closed after adding the record.
    • isPollingRequired

      public boolean isPollingRequired()
      Indicates if the shouldClose(RecordBuffer, BufferedReader, Record) method should be called every second for open buffers -- even if no new data is present -- to release any data that may have been collected (passing in a null record).
    • limitedTimeFromOpen

      public static BufferStrategy limitedTimeFromOpen(long milliseconds)
      Closes buffers after they've been open for the specified time.
    • limitedTimeFromLastUpdate

      public static BufferStrategy limitedTimeFromLastUpdate(long milliseconds)
      Closes buffers after the specified time has elapsed since the last record was added.
    • limitedTime

      public static BufferStrategy limitedTime(String fieldName, long milliseconds)
      Closes buffers after the specified datetime field has increased by a set amount of time since it was first seen. This is useful for aggregating hourly logs by their timestamp field.
      Parameters:
      fieldName - the datetime field to test
      milliseconds - the amount of time the field should increase before closing the buffer
    • limitedRecords

      public static BufferStrategy limitedRecords(long count)
      Closes the buffer after the specified amount of records have been collected.
    • scheduled

      public static BufferStrategy scheduled(Scheduler scheduler)
      Closes buffers on a set clock schedule. For example, every 30 minutes on the hour and half hour.
      See Also:
    • scheduled

      public static BufferStrategy scheduled(String fieldName, Scheduler scheduler)
      Closes buffers on a set clock schedule for the specified datetime field. For example, every 30 minutes at 15 and 45 past the hour. This is useful for aggregating hourly logs at set times by their timestamp field.
      Parameters:
      fieldName - the datetime field to test
      scheduler - the scheduler to use
      See Also:
    • hourlyScheduleInMinutes

      public static BufferStrategy hourlyScheduleInMinutes(int... minutesPastHour)
      Closes buffers at fixed minutes past each hour (for example: 0, 15, 30, and 45 minutes past each hour).
    • hourlyScheduleInMinutes

      public static BufferStrategy hourlyScheduleInMinutes(String fieldName, int... minutesPastHour)
      Closes buffers at fixed minutes past each hour for the specified datetime field (for example: 0, 15, 30, and 45 minutes past each hour).
      Parameters:
      fieldName - the datetime field to test
      minutesPastHour -
      Returns:
    • or

      public static BufferStrategy or(BufferStrategy... strategies)
      Closes buffers if any of the supplied strategies request it.
    • and

      public static BufferStrategy and(BufferStrategy... strategies)
      Closes buffers if all of the supplied strategies request it.
    • limitedTimeFromOpenOrLimitedRecords

      public static BufferStrategy limitedTimeFromOpenOrLimitedRecords(long milliseconds, long recordCount)
      Closes buffers after they've been open for the specified time or the specified amount of records have been collected.
    • limitedTimeFromLastUpdateOrLimitedRecords

      public static BufferStrategy limitedTimeFromLastUpdateOrLimitedRecords(long milliseconds, long recordCount)
      Closes buffers after the specified time has elapsed since the last record was added or the specified amount of records have been collected.