How to transfer files between Cloud Connections

Overview

This article covers how to move files between AWS S3 and Azure Blob Storage, via SFTP.

To achieve this, you will need an SFTP Gateway v3 server (which can be on either AWS or Azure). Then, using Cloud Connections, you point SFTP Gateway to:

1. An S3 bucket
2. A Blob Storage Container

Once this is done, you mount a Folder to each of these Cloud Connections. And then you point the SFTP user's Home Directory to a Folder that is a parent to the two previously mentioned Folder mappings.

When the SFTP user logs in, they will see both Folder mappings. And then you can simply drag a file from S3 to Blob, or vice-versa.

Before you begin

You will need SFTP Gateway v3 deployed in your AWS account.

Next, make sure that your Default Cloud Connection is configured properly. If you used CloudFormation, this should work automatically. Otherwise, here's an article with instructions on how to do this.

Note: It doesn't matter where Default Cloud Connection points to. It just needs to work, so the SFTP user has a file system to work with.

Create a Cloud Connection to AWS S3

First, create a Cloud Connection to AWS S3. Refer to this article in our AWS documentation for instructions on how to do this.

In this example, the Cloud Connection has the following properties:

• Connection Name: AWS Cloud Connection
• S3 URL: s3://blob-to-s3-example
• Region: us-east-1

Create a Cloud Connection to Azure Blob Storage

Next, create a Cloud Connection that points to Azure Blob Storage. Refer to this article for instructions.

In this example, the Cloud Connection has the following properties:

• Connection Name: Azure Cloud Connection
• Container Resource URI: https://bryce.blob.core.windows.net/blob-to-s3-example
  • bryce represents the Storage Account
  • blob-to-s3-example represents the Container
• Unique credentials: This is the Connection String of the Storage Account access key, Key 1

Map Folders to the Cloud Connections

In order to make use of Cloud Connections, you will need to map Folders to them.

Click on the Folders tab. The default Folder structure looks like this:

/
|--Users/  <-- SFTP home directories are created in here by default

Create a new folder named multi-cloud, and set it to inherited. Your folder structure should now look like this:

/
|--multi-cloud/
|--Users/  

Under the /multi-cloud Folder, create a new Folder named aws-folder that points to the AWS Cloud Connection.

Likewise, create a new Folder named azure-folder that points to the Azure Cloud Connection.

The structure of the folder system should look like this:

/
|--multi-cloud/
   |--aws-folder/    <-- maps to "AWS Cloud Connection"
   |--azure-folder/  <-- maps to "Azure Cloud Connection"
|--Users/

Set the SFTP user's Home Directory

Next, create a new SFTP user.

Under Home Directory, choose Select home directory.

Select the /multi-cloud Folder.

The SFTP user's Home Directory points to the /multi-cloud Folder, which is the parent of the two other Folders pointing to the AWS and Azure Cloud Connections.

This means that this SFTP user can access both clouds.

Transferring files

Finally, log in as the SFTP user via FileZilla.

You should see a chroot directory "/"

Underneath that, you should see a subfolder for each Cloud Connection.

AWS Cloud Connection Folder:

Azure Cloud Connection Folder:

To transfer a file, simply drag and drop it from one Folder to the other. In my case, I dragged files from Azure to AWS.

In this screenshot of S3, you see files originating from Azure.