Application backup of v2

Overview

One of the first steps toward migrating from v2 to v3 is performing an application backup of SFTP Gateway v2.

The application backup is similar to a database dump, and it contains all the SFTP users, their mappings to Blob storage destinations, and other settings. This data is structured in a yaml file, which is similar to JSON.

Some customers like to perform Azure backups on the VMs or take snapshots. While these can be helpful at times, the preferred approach is to make sure an SFTP Gateway application backup is taken. This backup will allow you to quickly recover in a new v2, or even v3 environment.

Is it safe to perform this backup on production?

In some organizations, the original person who set up version 2 is no longer in the same position. So managing version 2 is a mystery, and with that comes fear of doing anything in production.

Yes, it is safe to perform the application backup in production. In fact, this backup should be performed ideally on a weekly basis, or at least monthly. The biggest risk comes from neglecting to perform this backup, because you need it to quickly recover should something happen to v2 production.

While you are here

While we are on the topic of time-to-recovery for your v2 production, this is a good time to take a moment and monitor the health of your v2 instance.

Run the following command:

sudo su
df -h

This command shows the amount of storage available ("disk free, human readable"). You want to make sure you have ideally at least 50% of free space on your root / partition. If you are using more than 85% (or even 90%+), you should stop what you're doing and free up some disk space. See this KB article on managing disk space.

How does the backup work?

The backup is a Python script that you download from our public S3 bucket. This Python script executes the following command:

sftpgw.sh get-users

This command is part of our SFTP Gateway v2 command line utilities. This particular command lists out all SFTP users, along with their attributes.

The Python script iterates over each SFTP user object, and then starts constructing a yaml document that contains a user array.

The Python script also reads other files and settings on the VM, and combines this information in the yaml structure.

A note on passwords and public keys

Any passwords set on SFTP Gateway v2 will be stored in the yaml backup file in a hashed format. It is critical that you keep these yaml backup files in a secure location. Although you cannot easily infer the original password from the hash, it can be done theoretically.

Public keys on the other hand are not considered sensitive information. Public keys are designed to be exposed, and there is no way to use it to guess the private key.

Installing dependencies

If you are on an earlier version of SFTP Gateway v2 (e.g. 2.000.02.azu), you will need to first install some dependencies. There are Python modules needed by the backup script. For example, pyyaml is used for writing to yaml.

On later versions of 2.x, we pre-install these Python modules. But for earlier 2.x versions, you may need to install these manually:

sudo su
apt install -y python-pip
pip install pyyaml requests

These commands first install pip, which is a Python module installation package. Then, you use pip to install pyyaml and requests.

Download the Python backup script

To download the script, run the following commands:

sudo su
cd /root
wget http://www.sftpgateway.com/backup-and-recovery/backup.py

This downloads the Python backup script from our public S3 bucket, and saves it to your local file system. In this case, I am using the root user's home directory, but you can perform this in any Linux admin's home directory.

Run the Python script

To run the script, run these commands:

sudo su
python backup.py

The script generates a backup artifact named sftpgw.yml in the folder: /opt/sftpgw/backups/. Navigate to that location and verify that the backup file was generated.

The backup file is going to be in tar.gz format. So you want to extract the file using this command:

tar xzf sftpgw-*.tar.gz

You should see a .yml file appear. Inspect this file using the cat command:

cat sftpgw-*.yml

And verify that your SFTP users as well as their settings show up in the yaml document.

Copying the file off the VM

You can use a normal SFTP client like FileZilla or WinSCP to pull the backup file off your VM.

First, copy the backup artifact to your home directory (e.g. /home/azureuser/).

cd /opt/sftpgw/backups/
cp -a sftpgw-*.yml /home/azureuser/

Then, use FileZilla or WinSCP to log in as your Linux admin user (e.g. azureuser). Note: normally, you would use PuTTY to log in as azureuser. But this time, you will use WinSCP to do so.

One you are logged in, you can use the SFTP client to drag and drop the .yml file to your local workstation.