SFTP Gateway 2.0 Migration Guide
Note: This page applies to SFTP Gateway version 2.x. Visit Here for documentation on version 3.x.
SFTP Gateway 2.0 adds a number of improvements such as high availability and a web interface for user management.
If you are an existing SFTP Gateway customer running version 1.x, you can migrate your users and settings to a new SFTP Gateway 2.0 stack.
You can also migrate from one 2.0 stack to another. Use this to upgrade a single instance to high availability architecture. Or quickly spin up your setup in a different region as a cold failover.
This involves 3 main steps:
- Run an export script to generate a YAML file
- Transfer the YAML file to the new server
- Run an import script against the YAML file
Important Note: If you are transitioning from version 1.x to 2.0, you will need to add additional permissions to your IAM Role. Please see Post Configuration section of the SFTP Gateway 2.0 Usage Instructions - AMI article for policy creation and attachment instructions.
Export existing users
SSH into your old server, and run the following commands:
cd /home/ec2-user/
wget https://s3.amazonaws.com/thorntech-public-documents/sftpgateway/migration-scripts/sftpgw-export.py
sudo python sftpgw-export.py
This downloads and runs the export script, generating a new file: sftpgw.yml.
This YAML file contains your global settings, as well as the settings for each SFTP user.
Take a look, and make sure each setting is correct.
Transfer the YAML file
Download the YAML file from the old server to your local Mac:
scp -i <private.key> ec2-user@<old server public ip>:/home/ec2-user/sftpgw.yml .
Then, upload it to the new server:
scp -i <private.key> sftpgw.yml ec2-user@<new server public ip>:/home/ec2-user/
Import users into the new server
SSH into your new SFTP Gateway server, and run the following commands:
cd /home/ec2-user/
wget https://s3.amazonaws.com/thorntech-public-documents/sftpgateway/migration-scripts/sftpgw-import.py
sudo python sftpgw-import.py sftpgw.yml
This downloads the import script, and runs it against the sftpgw.yml file.
The output contains the commands used to create each user.
Caveats and gotchas
Single Thread
The sftpgw-export.py script sets sftpgateway.singlethread=no, regardless of the value on the original server.
In SFTP Gateway 1.003 and prior, we recommended setting sftpgateway.singlethread=yes.
This stabilizes the server, because it prevents multiple threads from overwhelming the server's memory.
Since the SFTP Gateway 1.004 update, we now recommend setting sftpgateway.singlethread=no.
This version lets you set the number of workers, so you can safely upload files faster.
Default Bucket
The sftpgw-export.py script migrates the sftpgateway.bucketname setting, overriding your existing default bucket name.
You can change it back using the web interface: click on Settings, update the field for the default bucket name, and click Apply.
SSE-C
If any user was configured with SSE-C in SFTP Gateway 1.x, the migration script changes the encryption option to SSE-S3.
SFTP Gateway 2.0 no longer supports the SSE-C option. SSE-C can be risky, since losing a key results in data loss. SSE-S3 and KMS are safer and easier options, so SSE-C should only be used if it's a hard requirement.
If you have S3 objects encrypted with SSE-C keys, wait until you decrypt/re-encrypt your S3 objects using SSE-S3 before continuing.
S3 bucket retention
By default, CloudFormation does not delete an S3 bucket when you delete a stack. This is a safety feature to prevent data loss.
You may want to delete your old SFTP Gateway 1.x after the migration. If you have any objects in your S3 bucket, the deletion will fail. Try deleting your stack a second time, and check the box to Retain the S3 bucket when prompted. The CloudFormation stack will finish deleting all the resources except for the S3 bucket.