Package com.northconcepts.datapipeline.internal.lang

Class FileSystem

java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.internal.lang.FileSystem
Direct Known Subclasses:
AmazonS3FileSystem, DropBoxFileSystem, FtpFileSystem, GoogleDriveFileSystem, HdfsFileSystem, SftpFileSystem, ZipFileSystem
public abstract class FileSystem extends Endpoint
Abstract class for reading and writing to virtual file systems.

  • Constructor Details

    • FileSystem

      public FileSystem()
      Creates the file system or throws a DataException if the edition or license does not allow file systems.

  • Method Details

    • getCredentialsResolver

      public CredentialsResolver getCredentialsResolver()
      Returns the resolver supplying this file system's credentials or null if none is set.

    • setCredentialsResolver

      public FileSystem setCredentialsResolver(CredentialsResolver credentialsResolver)
      Sets the resolver supplying this file system's credentials. Subclasses resolve it at the top of their Endpoint.open() (via resolveCredentials()) — never at construction — so rotated secrets are picked up on the next connection.

    • resolveCredentials

      protected Credentials resolveCredentials() throws DataException
      Resolves and snapshots credentials for this connection cycle. Subclasses call this at the top of their Endpoint.open(). Returns null when no resolver is set, letting subclasses fall back to ambient/role credentials or backend-specific configuration.

      The snapshot is an independent copy; close() wipes it without affecting the resolver's own instance. Any previous snapshot is wiped before resolving so a failed resolve never leaves stale credentials behind.

      Throws:
      DataException

    • refreshCredentials

      protected Credentials refreshCredentials() throws DataException
      Forces the resolver to re-fetch (see CredentialsResolver.refresh()), then resolves a fresh snapshot. For long-lived connections that need to re-authenticate mid-session — for example when Credentials.isExpired(long) reports the current snapshot is about to expire.
      Throws:
      DataException

    • getCurrentCredentials

      protected Credentials getCurrentCredentials()
      Returns the credentials snapshot resolved for this connection cycle or null if there isn't one.

    • credentialValue

      protected static String credentialValue(Credentials credentials, String key, String fallback)
      Returns the value for the given key from the resolved credentials or fallback when the credentials are null or don't contain the key. Lets connectors mix resolver-supplied secrets with constructor/setter-supplied values per key: 
       Credentials credentials = resolveCredentials();
 String userName = credentialValue(credentials, Credentials.USERNAME, this.userName);
 String password = credentialValue(credentials, Credentials.PASSWORD, this.password);

    • close

      public void close() throws DataException
      Wipes this connection cycle's credentials snapshot, then closes this endpoint.
      Overrides:
      close in class Endpoint
      Throws:
      DataException