Package com.northconcepts.datapipeline.parquet.file

Class ParquetOutputFile

java.lang.Object
com.northconcepts.datapipeline.parquet.file.ParquetOutputFile
public class ParquetOutputFile extends Object
A Parquet
invalid reference 
OutputFile
that writes to a Hadoop
invalid reference 
Path
, with checksum writing turned off on checksum file systems.

  • Method Details

    • getBlockFileSystems

      public static Set<String> getBlockFileSystems()
      Returns the URI schemes of the file systems treated as supporting block sizes (hdfs, webhdfs, viewfs).

    • fromPath

      public static ParquetOutputFile fromPath(Path path, Configuration conf) throws IOException
      Creates an output file for the path qualified by its file system, turning off checksum writing on that file system if it is a checksum file system.
      Throws:
      IOException

    • getConfiguration

      public Configuration getConfiguration()

    • create

      public PositionOutputStream create(long blockSizeHint) throws IOException
      Throws:
      IOException

    • createOrOverwrite

      public PositionOutputStream createOrOverwrite(long blockSizeHint) throws IOException
      Throws:
      IOException

    • supportsBlockSize

      public boolean supportsBlockSize()

    • defaultBlockSize

      public long defaultBlockSize()

    • getPath

      public String getPath()

    • toString

      public String toString()
      Overrides:
      toString in class Object