Read from FTP Using a Credentials Resolver

This example shows how to read a CSV file from an FTP server without a username or password in your code. The FtpFileSystem is given a SuppliedCredentialsResolver, which turns any Supplier<Credentials> into a CredentialsResolver. That one-line bridge is how you plug in a secret store such as HashiCorp Vault or AWS Secrets Manager; the lambda in the listing returns placeholders where a real job would call the store.

Because the supplier runs every time the file system opens, a rotated password is used on the next connection. When the lookup is slow or rate limited, wrap the resolver in a CachingCredentialsResolver as shown in Cache and Register a Credentials Resolver. The same approach works for Read from SFTP Using a Credentials Resolver.

Java Code Listing

package com.northconcepts.datapipeline.examples.security;

import java.io.InputStreamReader;

import com.northconcepts.datapipeline.core.DataReader;
import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.core.StreamWriter;
import com.northconcepts.datapipeline.csv.CSVReader;
import com.northconcepts.datapipeline.filesystem.ftp.FtpFileSystem;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.security.Credentials;
import com.northconcepts.datapipeline.security.SuppliedCredentialsResolver;

public class ReadFromFtpUsingACredentialsResolver {

    private static final String HOST = "localhost";
    private static final int PORT = 21;
    private static final String FOLDER = "/upload";
    private static final String FILE = "trades.csv";

    public static void main(String[] args) throws Throwable {
        FtpFileSystem ftp = new FtpFileSystem(HOST, PORT)
                .setCredentialsResolver(new SuppliedCredentialsResolver(() -> Credentials.of(
                        Credentials.USERNAME, "YOUR USERNAME",
                        Credentials.PASSWORD, "YOUR PASSWORD")));
        ftp.open();
        try {
            DataReader reader = new CSVReader(new InputStreamReader(ftp.readFile(FOLDER, FILE)))
                    .setFieldNamesInFirstRow(true);
            DataWriter writer = StreamWriter.newSystemOutWriter();

            Job.run(reader, writer);
        } finally {
            ftp.close();
        }
    }

}

Code Walkthrough

1. HOST, PORT, FOLDER and FILE identify the server (port 21) and the file to read.
2. A FtpFileSystem is created with the host and port only. setCredentialsResolver() is given a SuppliedCredentialsResolver wrapping a lambda that builds the login with Credentials.of() from key/value pairs: Credentials.USERNAME and Credentials.PASSWORD.
3. ftp.open() invokes the supplier and logs in with the username and password it returns.
4. ftp.readFile(FOLDER, FILE) returns an InputStream for the remote file, wrapped in a CSVReader with field names in the first row.
5. Job.run() transfers the records to a StreamWriter that prints them to the console.
6. ftp.close() in the finally block disconnects from the server.

Console Output

Each record in trades.csv is printed to the console, followed by the record count.