com.northconcepts.datapipeline.googledrive

Class GoogleDriveFileSystem

java.lang.Object

com.northconcepts.datapipeline.core.DataObject

com.northconcepts.datapipeline.core.Endpoint

com.northconcepts.datapipeline.internal.lang.FileSystem

com.northconcepts.datapipeline.googledrive.GoogleDriveFileSystem

public class GoogleDriveFileSystem
extends FileSystem

A FileSystem that reads and writes files stored in Google Drive.

Like every FileSystem, this class exposes raw file operations only — readFile, writeFile and listFolder/listRootFolder — and returns plain InputStream/OutputStream handles that an ordinary DataPipeline reader or writer operates on. It does not parse records itself. Because Google Drive identifies files by an opaque id rather than a path, the String name accessors resolve a name to the first matching file or folder; multiple entries may share a name but have distinct ids.

Authorization. Two credential modes are supported:

• Resolver-supplied OAuth access token — construct with GoogleDriveFileSystem(String) and register a CredentialsResolver via setCredentialsResolver(CredentialsResolver) that supplies Credentials.TOKEN. The interactive flow is skipped entirely and the token is re-resolved on each open(), so rotated tokens are picked up automatically.
• Interactive installed-app flow — construct with GoogleDriveFileSystem(String, String, InputStream, java.io.File) passing the client secrets JSON and a folder to persist the granted credentials. The first open() triggers a browser consent prompt; subsequent runs reuse the stored credentials.

Lifecycle. The constructor only stores configuration; open() builds the transport, resolves credentials and creates the Drive service. A write runs the upload on a background thread fed by a piped stream; close the returned OutputStream to let that thread finish, after which close() releases the piped streams and shuts the transport down. Note that close() does not itself block waiting for an in-flight upload — if the write thread has not yet signalled completion it only delegates to super.close(). Instances are not thread-safe and are intended to be used by a single pipeline at a time.

See Also:

FileSystem, CredentialsResolver

Field Summary

Fields

Modifier and Type	Field and Description
static String	GOOGLE_FOLDER_TYPE The Google Drive MIME type identifying an entry as a folder rather than a file.

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors

Constructor and Description
GoogleDriveFileSystem(String appName) Enables access to Google Drive with an OAuth access token supplied through a CredentialsResolver (see setCredentialsResolver(CredentialsResolver)) — no client secrets, credentials folder, or interactive authorization required.
GoogleDriveFileSystem(String appName, String userName, InputStream clientSecrets, File credentialsFolder) Enables access to Google Drive.

Method Summary

Modifier and Type	Method and Description
void	close() Closes the Google Drive connection.
String	getFileId(String fileName) Returns the first occurrence of the Google Drive File Id with the matching fileName.
String	getFolderId(String folderName) Returns the first occurrence of the Google Drive Folder Id with the matching folderName.
List<File>	listFolder(String folderName) Returns a List of all Google Drive files and folders under the specified folder.
List<File>	listRootFolder() Returns a List of all Google Drive files and folders.
void	open() Opens a connection to the Google Drive file system.
InputStream	readFile(String fileName) Reads the first occurrence of a Google Drive File with matching fileName.
InputStream	readFile(String folderName, String fileName) Reads the first occurrence of a Google Drive File with matching fileName in the specified folderName.
GoogleDriveFileSystem	setCredentialsResolver(CredentialsResolver credentialsResolver) Sets the resolver supplying this file system's OAuth access token (Credentials.TOKEN) — resolved on each open(), never at construction, so refreshed/rotated tokens are picked up on the next connection.
OutputStream	writeFile(String fileName) Writes the file to the Google Drive File System.
OutputStream	writeFile(String folderName, String fileName) Write the file to the specified folder.

Methods inherited from class com.northconcepts.datapipeline.internal.lang.FileSystem

credentialValue, getCredentialsResolver, getCurrentCredentials, refreshCredentials, resolveCredentials

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

addElapsedtime, addExceptionProperties, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getNestedEndpoint, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getRootEndpoint, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription, toString

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

GOOGLE_FOLDER_TYPE

public static final String GOOGLE_FOLDER_TYPE

The Google Drive MIME type identifying an entry as a folder rather than a file.

See Also:

Constant Field Values

Constructor Detail

GoogleDriveFileSystem

public GoogleDriveFileSystem(String appName,
                             String userName,
                             InputStream clientSecrets,
                             File credentialsFolder)

Enables access to Google Drive.

Parameters:

appName - application name to use when connecting to Google Drive

userName - the username of the Google Drive account

clientSecrets - the JSON file client secret provided by Google Drive

credentialsFolder - the folder to store the Google Drive Credentials

GoogleDriveFileSystem

public GoogleDriveFileSystem(String appName)

Enables access to Google Drive with an OAuth access token supplied through a CredentialsResolver (see setCredentialsResolver(CredentialsResolver)) — no client secrets, credentials folder, or interactive authorization required.

Parameters:

appName - application name to use when connecting to Google Drive

Method Detail

setCredentialsResolver

public GoogleDriveFileSystem setCredentialsResolver(CredentialsResolver credentialsResolver)

Sets the resolver supplying this file system's OAuth access token (Credentials.TOKEN) — resolved on each open(), never at construction, so refreshed/rotated tokens are picked up on the next connection. When a token is resolved, the interactive authorization flow (client secrets + stored credentials + browser consent) is skipped entirely; when it isn't, the constructor-configured flow runs unchanged.

Overrides:

setCredentialsResolver in class FileSystem

open

public void open()
          throws DataException

Opens a connection to the Google Drive file system.

Overrides:

open in class Endpoint

Throws:

DataException

close

public void close()
           throws DataException

Closes the Google Drive connection.

Overrides:

close in class FileSystem

Throws:

DataException

listFolder

public List<File> listFolder(String folderName)
                      throws Throwable

Returns a List of all Google Drive files and folders under the specified folder.

Parameters:

folderName - name of the folder

Returns:

a List<File> of the folder's immediate children (possibly empty if the folder has none), or null if folderName is empty

Throws:

Throwable - if the folder does not exist or the Drive request fails

listRootFolder

public List<File> listRootFolder()
                          throws Throwable

Returns a List of all Google Drive files and folders. Including files in the trash.

Returns:

the contents as a List<File>

Throws:

Throwable

readFile

public InputStream readFile(String fileName)
                     throws Throwable

Reads the first occurrence of a Google Drive File with matching fileName. It's possible that multiple files will have the same file name but different file ids.

Parameters:

fileName - the file name to read.

Returns:

an InputStream

Throws:

Throwable

readFile

public InputStream readFile(String folderName,
                            String fileName)
                     throws Throwable

Reads the first occurrence of a Google Drive File with matching fileName in the specified folderName. It's possible that multiple files will have the same file name but different file ids.

Parameters:

folderName - the folder the file belongs to

fileName - the name of the file

Returns:

an InputStream

Throws:

Throwable

writeFile

public OutputStream writeFile(String fileName)
                       throws Throwable

Writes the file to the Google Drive File System. It's possible that multiple files will have the same file name but different file ids.

Parameters:

fileName - the name of the file

Returns:

an OutputStream for writing

Throws:

Throwable

writeFile

public OutputStream writeFile(String folderName,
                              String fileName)
                       throws Throwable

Write the file to the specified folder. It's possible in Google Drive that multiple files or folders will have the same name but different file ids.

Parameters:

folderName - the name of the folder

fileName - the name of the file

Returns:

an OutputStream for writing

Throws:

Throwable

getFolderId

public String getFolderId(String folderName)
                   throws Throwable

Returns the first occurrence of the Google Drive Folder Id with the matching folderName.

Parameters:

folderName - the name of the folder to search for

Returns:

the folder id of the first occurrence of folderName, or null if folderName is empty

Throws:

Throwable - if no folder named folderName exists or the Drive request fails

getFileId

public String getFileId(String fileName)
                 throws Throwable

Returns the first occurrence of the Google Drive File Id with the matching fileName.

Parameters:

fileName - the name of the file to search for

Returns:

the file id of the first occurrence of fileName, or null if fileName is empty

Throws:

Throwable - if no file named fileName exists or the Drive request fails