Package com.northconcepts.datapipeline.googledrive

Class GoogleDriveFileSystem

java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.internal.lang.FileSystem
com.northconcepts.datapipeline.googledrive.GoogleDriveFileSystem
public class GoogleDriveFileSystem extends FileSystem
A FileSystem that reads and writes files stored in Google Drive.

Like every FileSystem, this class exposes raw file operations only — readFile, writeFile and listFolder/listRootFolder — and returns plain InputStream/OutputStream handles that an ordinary DataPipeline reader or writer operates on. It does not parse records itself. Because Google Drive identifies files by an opaque id rather than a path, the String name accessors resolve a name to the first matching file or folder; multiple entries may share a name but have distinct ids.

Authorization. Two credential modes are supported:

Lifecycle. The constructor only stores configuration; open() builds the transport, resolves credentials and creates the Drive service. A write runs the upload on a background thread fed by a piped stream; close the returned OutputStream to let that thread finish, after which close() releases the piped streams and shuts the transport down. Note that close() does not itself block waiting for an in-flight upload — if the write thread has not yet signalled completion it only delegates to super.close(). Instances are not thread-safe and are intended to be used by a single pipeline at a time.

See Also:

  • Field Details

    • GOOGLE_FOLDER_TYPE

      public static final String GOOGLE_FOLDER_TYPE
      The Google Drive MIME type identifying an entry as a folder rather than a file.
      See Also:

  • Constructor Details

    • GoogleDriveFileSystem

      public GoogleDriveFileSystem(String appName, String userName, InputStream clientSecrets, File credentialsFolder)
      Enables access to Google Drive.
      Parameters:
      appName - application name to use when connecting to Google Drive
      userName - the username of the Google Drive account
      clientSecrets - the JSON file client secret provided by Google Drive
      credentialsFolder - the folder to store the Google Drive Credentials

    • GoogleDriveFileSystem

      public GoogleDriveFileSystem(String appName)
      Enables access to Google Drive with an OAuth access token supplied through a CredentialsResolver (see setCredentialsResolver(CredentialsResolver)) — no client secrets, credentials folder, or interactive authorization required.
      Parameters:
      appName - application name to use when connecting to Google Drive

  • Method Details

    • setCredentialsResolver

      public GoogleDriveFileSystem setCredentialsResolver(CredentialsResolver credentialsResolver)
      Sets the resolver supplying this file system's OAuth access token (Credentials.TOKEN) — resolved on each open(), never at construction, so refreshed/rotated tokens are picked up on the next connection. When a token is resolved, the interactive authorization flow (client secrets + stored credentials + browser consent) is skipped entirely; when it isn't, the constructor-configured flow runs unchanged.
      Overrides:
      setCredentialsResolver in class FileSystem

    • open

      public void open() throws DataException
      Opens a connection to the Google Drive file system.
      Overrides:
      open in class Endpoint
      Throws:
      DataException

    • close

      public void close() throws DataException
      Closes the Google Drive connection.
      Overrides:
      close in class FileSystem
      Throws:
      DataException

    • listFolder

      public List<File> listFolder(String folderName) throws Throwable
      Returns a List of all Google Drive files and folders under the specified folder.
      Parameters:
      folderName - name of the folder
      Returns:
      a List<File> of the folder's immediate children (possibly empty if the folder has none), or null if folderName is empty
      Throws:
      Throwable - if the folder does not exist or the Drive request fails

    • listRootFolder

      public List<File> listRootFolder() throws Throwable
      Returns a List of all Google Drive files and folders. Including files in the trash.
      Returns:
      the contents as a List<File>
      Throws:
      Throwable

    • readFile

      public InputStream readFile(String fileName) throws Throwable
      Reads the first occurrence of a Google Drive File with matching fileName. It's possible that multiple files will have the same file name but different file ids.
      Parameters:
      fileName - the file name to read.
      Returns:
      an InputStream
      Throws:
      Throwable

    • readFile

      public InputStream readFile(String folderName, String fileName) throws Throwable
      Reads the first occurrence of a Google Drive File with matching fileName in the specified folderName. It's possible that multiple files will have the same file name but different file ids.
      Parameters:
      folderName - the folder the file belongs to
      fileName - the name of the file
      Returns:
      an InputStream
      Throws:
      Throwable

    • writeFile

      public OutputStream writeFile(String fileName) throws Throwable
      Writes the file to the Google Drive File System. It's possible that multiple files will have the same file name but different file ids.
      Parameters:
      fileName - the name of the file
      Returns:
      an OutputStream for writing
      Throws:
      Throwable

    • writeFile

      public OutputStream writeFile(String folderName, String fileName) throws Throwable
      Write the file to the specified folder. It's possible in Google Drive that multiple files or folders will have the same name but different file ids.
      Parameters:
      folderName - the name of the folder
      fileName - the name of the file
      Returns:
      an OutputStream for writing
      Throws:
      Throwable

    • getFolderId

      public String getFolderId(String folderName) throws Throwable
      Returns the first occurrence of the Google Drive Folder Id with the matching folderName.
      Parameters:
      folderName - the name of the folder to search for
      Returns:
      the folder id of the first occurrence of folderName, or null if folderName is empty
      Throws:
      Throwable - if no folder named folderName exists or the Drive request fails

    • getFileId

      public String getFileId(String fileName) throws Throwable
      Returns the first occurrence of the Google Drive File Id with the matching fileName.
      Parameters:
      fileName - the name of the file to search for
      Returns:
      the file id of the first occurrence of fileName, or null if fileName is empty
      Throws:
      Throwable - if no file named fileName exists or the Drive request fails