com.northconcepts.datapipeline.internal.lang

Class FileSystem

java.lang.Object

com.northconcepts.datapipeline.core.DataObject

com.northconcepts.datapipeline.core.Endpoint

com.northconcepts.datapipeline.internal.lang.FileSystem

Direct Known Subclasses:

AmazonS3FileSystem, DropBoxFileSystem, FtpFileSystem, GoogleDriveFileSystem, HdfsFileSystem, SftpFileSystem, ZipFileSystem

public abstract class FileSystem
extends Endpoint

Abstract class for reading and writing to virtual file systems.

Field Summary

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors

Constructor and Description
FileSystem()

Method Summary

Modifier and Type	Method and Description
void	close() Wipes this connection cycle's credentials snapshot, then closes this endpoint.
protected static String	credentialValue(Credentials credentials, String key, String fallback) Returns the value for the given key from the resolved credentials or fallback when the credentials are null or don't contain the key.
CredentialsResolver	getCredentialsResolver()
protected Credentials	getCurrentCredentials() Returns the credentials snapshot resolved for this connection cycle or null if there isn't one.
protected Credentials	refreshCredentials() Forces the resolver to re-fetch (see CredentialsResolver.refresh()), then resolves a fresh snapshot.
protected Credentials	resolveCredentials() Resolves and snapshots credentials for this connection cycle.
FileSystem	setCredentialsResolver(CredentialsResolver credentialsResolver) Sets the resolver supplying this file system's credentials.

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

addElapsedtime, addExceptionProperties, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getNestedEndpoint, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getRootEndpoint, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, open, setCaptureElapsedTime, setDescription, toString

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FileSystem

public FileSystem()

Method Detail

getCredentialsResolver

public CredentialsResolver getCredentialsResolver()

setCredentialsResolver

public FileSystem setCredentialsResolver(CredentialsResolver credentialsResolver)

Sets the resolver supplying this file system's credentials. Subclasses resolve it at the top of their Endpoint.open() (via resolveCredentials()) — never at construction — so rotated secrets are picked up on the next connection.

resolveCredentials

protected Credentials resolveCredentials()
                                  throws DataException

Resolves and snapshots credentials for this connection cycle. Subclasses call this at the top of their Endpoint.open(). Returns null when no resolver is set, letting subclasses fall back to ambient/role credentials or backend-specific configuration.

The snapshot is an independent copy; close() wipes it without affecting the resolver's own instance. Any previous snapshot is wiped before resolving so a failed resolve never leaves stale credentials behind.

Throws:

DataException

refreshCredentials

protected Credentials refreshCredentials()
                                  throws DataException

Forces the resolver to re-fetch (see CredentialsResolver.refresh()), then resolves a fresh snapshot. For long-lived connections that need to re-authenticate mid-session — for example when Credentials.isExpired(long) reports the current snapshot is about to expire.

Throws:

DataException

getCurrentCredentials

protected Credentials getCurrentCredentials()

Returns the credentials snapshot resolved for this connection cycle or null if there isn't one.

credentialValue

protected static String credentialValue(Credentials credentials,
                                        String key,
                                        String fallback)

Returns the value for the given key from the resolved credentials or fallback when the credentials are null or don't contain the key. Lets connectors mix resolver-supplied secrets with constructor/setter-supplied values per key:

 Credentials credentials = resolveCredentials();
 String userName = credentialValue(credentials, Credentials.USERNAME, this.userName);
 String password = credentialValue(credentials, Credentials.PASSWORD, this.password);
 

close

public void close()
           throws DataException

Wipes this connection cycle's credentials snapshot, then closes this endpoint.

Overrides:

close in class Endpoint

Throws:

DataException