Package com.northconcepts.datapipeline.foundations.pipeline.dataset

Class MemoryDataset

java.lang.Object
com.northconcepts.datapipeline.foundations.core.Bean
com.northconcepts.datapipeline.foundations.core.FoundationObject
com.northconcepts.datapipeline.foundations.pipeline.dataset.Dataset
com.northconcepts.datapipeline.foundations.pipeline.dataset.MemoryDataset
All Implemented Interfaces:
DataExceptionContributor, JsonSerializable, RecordSerializable, XmlSerializable, Closeable, Serializable, AutoCloseable, Iterable<Record>
public class MemoryDataset extends Dataset
Caches the dataset's records in RAM.
See Also:

  • Constructor Details

    • MemoryDataset

      public MemoryDataset(AbstractPipeline pipeline)
      Creates a dataset that keeps the given pipeline's records in memory once Dataset.load() is called.

  • Method Details

    • setPipeline

      public MemoryDataset setPipeline(AbstractPipeline pipeline)
      Overrides:
      setPipeline in class Dataset

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Overrides:
      close in class Dataset

    • getRecordCount

      public long getRecordCount()
      Description copied from class: Dataset
      Returns the number of records loaded into this dataset so far.
      Specified by:
      getRecordCount in class Dataset

    • getRecord

      public Record getRecord(long index)
      Description copied from class: Dataset
      Returns the loaded record at the given 0-based index.
      Specified by:
      getRecord in class Dataset

    • getColumnCount

      public long getColumnCount()
      Specified by:
      getColumnCount in class Dataset

    • getColumnNames

      public List<String> getColumnNames()
      Specified by:
      getColumnNames in class Dataset

    • getColumn

      public Column getColumn(int index)
      Specified by:
      getColumn in class Dataset

    • getColumn

      public Column getColumn(String name)
      Description copied from class: Dataset
      Returns the column with the given name or null if there is none.
      Specified by:
      getColumn in class Dataset

    • getOrCreateColumn

      protected Column getOrCreateColumn(String name, int index)
      Description copied from class: Dataset
      Returns the named column, creating it with the given 0-based field index if needed; called while collecting column stats.
      Specified by:
      getOrCreateColumn in class Dataset

    • getColumns

      public List<Column> getColumns()
      Specified by:
      getColumns in class Dataset

    • beforeLoad

      protected void beforeLoad()
      Description copied from class: Dataset
      Called at the start of the data loading process, but before any records or column stats have been loaded.
      Specified by:
      beforeLoad in class Dataset

    • afterLoad

      protected void afterLoad()
      Description copied from class: Dataset
      Called at the end of the data loading process after all the records and column stats have been loaded.
      Specified by:
      afterLoad in class Dataset

    • createDataWriter

      protected DataWriter createDataWriter()
      Description copied from class: Dataset
      Writes records to this dataset's cache after clearing it.
      Specified by:
      createDataWriter in class Dataset