Azure SFTP Gateway Backup and Recovery
We have developed the following process for recovering SFTP Gateway users and configurations. This will be useful in the following cases:
- A region outage requires you to do a cold failover to another region
- You need to migrate or replicate to another region
- An SFTP Gateway instance is accidentally deleted
Note: The backup and recovery process captures the credentials for both password and SSH key based authentication.
Make sure you test this process thoroughly in your own environment.
Disclaimer: This backup and recovery strategy is intended for standard installations of SFTP Gateway. If you have customized your SFTP Gateway installation(s), please contact us at azure-support@thorntech.com to see if this solution will work for you.
Backup
Creating a backup
The SFTP Gateway backup script is a Python script that you install on your SFTP Gateway instance. The script backs up
the following data to a single flat YAML file (and later gets compressed as a tar.gz
file).
- user properties
- user passwords/keys
- global SFTP Gateway properties
This backup artifact can be imported into a new SFTP Gateway stack to restore settings and configuration.
The sftpgw-backup.py
script will create a backup file named sftpgw-backup-YYYY-MM-DD-HH-MM-SS.tar.gz
. By default,
this file is saved in the/opt/sftpgw/backups
directory (which gets created if it doesn't already exist).
To run the backup script:
SSH to the server as the administrator user (this is generally
ubuntu
)Install pip and script dependencies
sudo apt install -y python-pip pip install PyYaml
Download the backup script
cd /home/ubuntu/ wget https://s3.amazonaws.com/thorntech-public-documents/sftpgateway/backup-and-recovery/sftpgw-backup.py
Run the backup script
sudo python sftpgw-backup.py
This will create a backup file
sftpgw-YYYY-MM-DD-HH-MM-SS.tar.gz
. This will later be used to recover SFTP Gateway users and configurations.(Optional) Use the
--destination
parameter to save the backup artifact to a specific location.sudo python sftpgw-backup.py --destination /path/to/backup/directory/
This parameter will create the directory if it doesn't exist. The script will fail if the destination is a file.
Next steps
Once a backup artifact has been created, you can copy it to Blog storage or some other external location. This can be done using a "backup" SFTP user and saving the file to that user's uploads directory.
To copy the backup artifact using SFTP Gateway:
- Create a "backup" user with the web interface, CLI, or API
- You can point this user's uploads to a specific Blob storage container
- Run the backup script with the backup user's
/home/bacup/home/backup/uploads
directory as the destinationsudo python sftpgw-backup.py --destination /home/backup/home/backup/uploads
This will upload the backup file to Blob storage as soon as it is created. Storing the backup artifact on Blob storage will improve its availability should something happen to the SFTP Gateway instance.
Alternatively, you can copy the backup artifact to an external location manually, or in an automated fashion.
Schedule backup
A scheduled backup can help improve business continuity since your user settings will be up to date as far as the most recent backup.
To schedule a backup, you can create a cron job to run the backup script.
Here is an example for running the backup at 11:00 PM
every Friday:
Open the root crontab for editing. This will open the crontab in vim[1]
sudo crontab -e
Enter the following as a new line
0 23 * * Fri PATH=$PATH:/usr/local/bin; python /home/ubuntu/sftpgw-backup.py --destination /home/backup/home/backup/uploads >> /var/log/sftpgw/backup.log 2>&1
This is a breakdown of the syntax:
0 23 * * Fri
- Time schedule that runs script at 23:00 server time every Friday.PATH=$PATH:/usr/local/bin;
- Adds/usr/local/bin
to the working path for this cronjob.python /home/ubuntu/sftpgw-backup.py --destination /home/backup/home/backup/uploads
- Runs the script and writes the backup files to/home/backup/home/backup/uploads
.>> /var/log/sftpgw/backup.log 2>&1
- Directs all output to a specific log.
Here is a good resource for building cronjob schedules - https://crontab.guru/
Recovery
To recover SFTP Gateway users and configurations:
SSH to the server as the administrator user (this is generally
ubuntu
)Install pip and script dependencies
sudo apt install -y python-pip pip install PyYaml
Download the recovery script:
cd /home/ubuntu/ wget https://s3.amazonaws.com/thorntech-public-documents/sftpgateway/backup-and-recovery/sftpgw-recovery.py
Copy the
sftpgw-backup-YYYY-MM-DD-HH-MM-SS.yml.tar.gz
file to the server. For example, if the backup file is stored in Blob stoarge, you can use theaz storage blob download
command:account_name=$(sudo grep storage.account-name /opt/sftpgw/application.properties | cut -d'=' -f2) account_key=$(sudo grep storage.account-key /opt/sftpgw/application.properties | cut -d'=' -f2-) sudo az storage blob download --account-name $account_name --account-key $account_key --container-name <container_name> --name <prefix/to/cloud/backup/artifact> --file <path/to/server/download/file>
Run the python recovery script
sudo python sftpgw-recovery.py sftpgw-YYYY-MM-DD-HH-MM-SS.tar.gz
Your new instance will be in the same state as the original instance, including your users and SFTP Gateway configuration.
If you are unfamiliar with the text editor vim, here is a good resource to get you started - https://learnxinyminutes.com/docs/vim/ ↩