Class GenerateEntityFromDataset

java.lang.Object
com.northconcepts.datapipeline.foundations.tools.GenerateEntityFromDataset

public class GenerateEntityFromDataset extends Object
A utility class for generating EntityDef code from a DataReader.
  • Constructor Details

    • GenerateEntityFromDataset

      public GenerateEntityFromDataset()
  • Method Details

    • generateEntity

      public EntityDef generateEntity(DataReader reader)
      Samples the reader until getMinRecords() records are loaded or the wait time runs out, then returns an entity with a field for each column.
    • createEntity

      public EntityDef createEntity(Dataset dataset)
      Returns an entity with a field for each column of the loaded dataset, marking fields required when the column had no null or blank values.
    • createFieldDef

      public FieldDef createFieldDef(Column column)
      Returns a field definition typed according to getFieldTypeSelectionMode(); text fields get a maximum length rounded up to the next multiple of 100.
    • getMinRecords

      public long getMinRecords()
      Returns the number of records to sample before generating the entity (defaults to 1000).
    • setMinRecords

      public GenerateEntityFromDataset setMinRecords(long minRecords)
      Sets the number of records to sample before generating the entity; must be positive (defaults to 1000).
    • getMaxWaitTimeMillis

      public long getMaxWaitTimeMillis()
      Returns the maximum milliseconds to wait for the sample records to load, and again for their column statistics (defaults to 10000).
    • setMaxWaitTimeMillis

      public GenerateEntityFromDataset setMaxWaitTimeMillis(long maxWaitTimeMillis)
      Sets the maximum milliseconds to wait for the sample records to load, and again for their column statistics; must be positive (defaults to 10000).
    • getEntityName

      public String getEntityName()
    • setEntityName

      public GenerateEntityFromDataset setEntityName(String entityName)
    • getColumnStatsReaderThreads

      public int getColumnStatsReaderThreads()
      Indicates how many threads should be used to read and analyze the data in each column (defaults to 2).
    • setColumnStatsReaderThreads

      public GenerateEntityFromDataset setColumnStatsReaderThreads(int columnStatsReaderThreads)
      Indicates how many threads should be used to read and analyze the data in each column (defaults to 2).
    • isUseInferredType

      @Deprecated public boolean isUseInferredType()
      Deprecated.
      Indicates if the most frequently inferred type should be used as the field type instead of the best fit of the actual types (defaults to true).

      * For example: if a column contains strings which appears to be dates the inferred field type for that column will be date and the best fit would be string.

    • setUseInferredType

      @Deprecated public GenerateEntityFromDataset setUseInferredType(boolean useInferredType)
      Indicates if the most frequently inferred type should be used as the field type instead of the best fit of the actual types (defaults to true).

      * For example: if a column contains strings which appears to be dates the inferred field type for that column will be date and the best fit would be string.

    • getFieldTypeSelectionMode

      public GenerateEntityFromDataset.FieldTypeSelectionMode getFieldTypeSelectionMode()
      Gets the field type selection mode.

      The default is GenerateEntityFromDataset.FieldTypeSelectionMode.INFERRED.

    • setFieldTypeSelectionMode

      public GenerateEntityFromDataset setFieldTypeSelectionMode(GenerateEntityFromDataset.FieldTypeSelectionMode fieldTypeSelectionMode)
      Indicates how the field type should be selected for each column.

      The default is GenerateEntityFromDataset.FieldTypeSelectionMode.INFERRED.