Package com.northconcepts.datapipeline.amazons3

Class AmazonS3File

java.lang.Object
com.northconcepts.datapipeline.foundations.core.Bean
com.northconcepts.datapipeline.amazons3.AmazonS3File
All Implemented Interfaces:
DataExceptionContributor, JsonSerializable, RecordSerializable, XmlSerializable, Serializable
Direct Known Subclasses:
AmazonS3FileSink, AmazonS3FileSource
public abstract class AmazonS3File extends Bean
Base class for a file in Amazon S3, located by bucket name and path and accessed through an AmazonS3FileSystem that other files may share. Its record and XML forms keep the name, bucket, path, region, endpoint override and debug flag, but not credentials.
See Also:

  • Field Details

  • Constructor Details

    • AmazonS3File

      public AmazonS3File()
      Creates a file with its own new AmazonS3FileSystem.

    • AmazonS3File

      public AmazonS3File(AmazonS3FileSystem amazonS3FileSystem)
      Creates a file that uses the given file system, or a new one if it is null.

  • Method Details

    • assertValid

      protected void assertValid()
      Throws a DataException if this file has no AmazonS3FileSystem.

    • getName

      public String getName()

    • setName

      public AmazonS3File setName(String name)

    • getUrlPath

      public String getUrlPath()

    • getFileSize

      public long getFileSize()
      Returns the size in S3 in bytes, or -1 if the key is absent (other errors throw); a known size is kept until the bucket, path, or file system changes, a sink upload finishes, or refreshFileSize() is called.

    • refreshFileSize

      public long refreshFileSize()
      Looks the size up in S3 again, for an object that may have changed since its size was kept.

    • retainFileSize

      protected void retainFileSize(AmazonS3FileSystem fileSystem, String bucket, String key, Long size)
      Retains size only if this file still points at the object it was looked up for.

    • clearFileSize

      protected void clearFileSize()
      Forgets the kept file size so the next getFileSize() looks it up in S3 again.

    • getContentType

      public String getContentType()
      Returns null; the S3 object's content type is not looked up.

    • getFileType

      public FileType getFileType()

    • isLocal

      public boolean isLocal()

    • getCredentialsResolver

      public CredentialsResolver getCredentialsResolver()
      Returns the credentials resolver of this file's AmazonS3FileSystem or null if there is none.

    • setCredentialsResolver

      public AmazonS3File setCredentialsResolver(CredentialsResolver credentialsResolver)
      Sets the backend-neutral resolver supplying this file's credentials — resolved on each connection, so rotated secrets are picked up automatically. Takes precedence over setCredentialsProvider(AwsCredentialsProvider) and the default provider chain.

    • getCredentialsProvider

      public AwsCredentialsProvider getCredentialsProvider()
      Returns the AWS SDK credentials provider that this file's AmazonS3FileSystem uses when no CredentialsResolver is set.

    • setCredentialsProvider

      public AmazonS3File setCredentialsProvider(AwsCredentialsProvider credentialsProvider)
      Sets the AWS SDK credentials provider that this file's AmazonS3FileSystem uses when no CredentialsResolver is set.

    • useProfileCredentialsProvider

      public AmazonS3File useProfileCredentialsProvider(String profileName)
      Authenticates this file's AmazonS3FileSystem using the named profile from the AWS profile files, unless a CredentialsResolver is set, which takes precedence.

    • setBasicAWSCredentials

      public AmazonS3File setBasicAWSCredentials(String accessKey, String secretKey)
      Authenticates this file's AmazonS3FileSystem with the given access key and secret key, replacing its credentials provider, unless a CredentialsResolver is set, which takes precedence.

    • getAmazonS3FileSystem

      public AmazonS3FileSystem getAmazonS3FileSystem()

    • setAmazonS3FileSystem

      public AmazonS3File setAmazonS3FileSystem(AmazonS3FileSystem amazonS3FileSystem)

    • isDebug

      public boolean isDebug()
      Indicates if this file's AmazonS3FileSystem logs multipart upload progress at debug level.

    • setDebug

      public AmazonS3File setDebug(boolean debug)
      Indicates if this file's AmazonS3FileSystem logs multipart upload progress at debug level.

    • getBucketName

      public String getBucketName()

    • setBucketName

      public AmazonS3File setBucketName(String bucketName)

    • getPath

      public String getPath()
      Returns the key of this file's object within its bucket.

    • setPath

      public AmazonS3File setPath(String path)
      Sets the key of this file's object within its bucket.

    • toRecord

      public Record toRecord()
      Description copied from interface: RecordSerializable
      Converts this object's state to a record that RecordSerializable.fromRecord(Record) can load.
      Specified by:
      toRecord in interface RecordSerializable
      Overrides:
      toRecord in class Bean

    • fromRecord

      public AmazonS3File fromRecord(Record source)
      Description copied from interface: RecordSerializable
      Loads this instance's state from a record and returns this (for fluid API call chaining). For fluid API call chaining, the overridden method should change the declared return type to its class.
      Specified by:
      fromRecord in interface RecordSerializable
      Overrides:
      fromRecord in class Bean
      Parameters:
      source -
      Returns:
      this instance.

    • toXmlElement

      public Element toXmlElement(Document document)
      Description copied from interface: XmlSerializable
      Returns an element, created with document, describing this object; the default implementation throws a DataException.

    • fromXmlElement

      public AmazonS3File fromXmlElement(Element element)