Class GenerateEntityFromDataset
java.lang.Object
com.northconcepts.datapipeline.foundations.tools.GenerateEntityFromDataset
A utility class for generating EntityDef code from a DataReader.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic enumEnumeration of field type selection modes for determining how to select field types from column statistics. -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptioncreateEntity(Dataset dataset) Returns an entity with a field for each column of the loaded dataset, marking fields required when the column had no null or blank values.createFieldDef(Column column) Returns a field definition typed according togetFieldTypeSelectionMode(); text fields get a maximum length rounded up to the next multiple of 100.generateEntity(DataReader reader) Samples the reader untilgetMinRecords()records are loaded or the wait time runs out, then returns an entity with a field for each column.intIndicates how many threads should be used to read and analyze the data in each column (defaults to 2).Gets the field type selection mode.longReturns the maximum milliseconds to wait for the sample records to load, and again for their column statistics (defaults to 10000).longReturns the number of records to sample before generating the entity (defaults to 1000).booleanDeprecated.setColumnStatsReaderThreads(int columnStatsReaderThreads) Indicates how many threads should be used to read and analyze the data in each column (defaults to 2).setEntityName(String entityName) setFieldTypeSelectionMode(GenerateEntityFromDataset.FieldTypeSelectionMode fieldTypeSelectionMode) Indicates how the field type should be selected for each column.setMaxWaitTimeMillis(long maxWaitTimeMillis) Sets the maximum milliseconds to wait for the sample records to load, and again for their column statistics; must be positive (defaults to 10000).setMinRecords(long minRecords) Sets the number of records to sample before generating the entity; must be positive (defaults to 1000).setUseInferredType(boolean useInferredType) Deprecated.UsesetFieldTypeSelectionMode(FieldTypeSelectionMode)instead.
-
Constructor Details
-
GenerateEntityFromDataset
public GenerateEntityFromDataset()
-
-
Method Details
-
generateEntity
Samples the reader untilgetMinRecords()records are loaded or the wait time runs out, then returns an entity with a field for each column. -
createEntity
Returns an entity with a field for each column of the loaded dataset, marking fields required when the column had no null or blank values. -
createFieldDef
Returns a field definition typed according togetFieldTypeSelectionMode(); text fields get a maximum length rounded up to the next multiple of 100. -
getMinRecords
public long getMinRecords()Returns the number of records to sample before generating the entity (defaults to 1000). -
setMinRecords
Sets the number of records to sample before generating the entity; must be positive (defaults to 1000). -
getMaxWaitTimeMillis
public long getMaxWaitTimeMillis()Returns the maximum milliseconds to wait for the sample records to load, and again for their column statistics (defaults to 10000). -
setMaxWaitTimeMillis
Sets the maximum milliseconds to wait for the sample records to load, and again for their column statistics; must be positive (defaults to 10000). -
getEntityName
-
setEntityName
-
getColumnStatsReaderThreads
public int getColumnStatsReaderThreads()Indicates how many threads should be used to read and analyze the data in each column (defaults to 2). -
setColumnStatsReaderThreads
Indicates how many threads should be used to read and analyze the data in each column (defaults to 2). -
isUseInferredType
Deprecated.UsegetFieldTypeSelectionMode()instead.Indicates if the most frequently inferred type should be used as the field type instead of the best fit of the actual types (defaults to true).* For example: if a column contains strings which appears to be dates the inferred field type for that column will be date and the best fit would be string.
-
setUseInferredType
Deprecated.UsesetFieldTypeSelectionMode(FieldTypeSelectionMode)instead.Indicates if the most frequently inferred type should be used as the field type instead of the best fit of the actual types (defaults to true).* For example: if a column contains strings which appears to be dates the inferred field type for that column will be date and the best fit would be string.
-
getFieldTypeSelectionMode
Gets the field type selection mode.The default is
GenerateEntityFromDataset.FieldTypeSelectionMode.INFERRED. -
setFieldTypeSelectionMode
public GenerateEntityFromDataset setFieldTypeSelectionMode(GenerateEntityFromDataset.FieldTypeSelectionMode fieldTypeSelectionMode) Indicates how the field type should be selected for each column.The default is
GenerateEntityFromDataset.FieldTypeSelectionMode.INFERRED.
-
getFieldTypeSelectionMode()instead.