Running the In-Place Upgrade Script

Overview

This article goes over how to run the in-place upgrade script to upgrade to version 1.001.00 of StorageLink.

Important Mentions

Make a new Backup File

Our recommended approach for upgrading to a new version of StorageLink is to spin up a new StorageLink instance and import your Users & Settings into the new instance using a Backup file. You can export a new Backup file under Settings ---> Backup & Recovery ---> Export Backup File. For more information on our recommended upgrade process, check out this article.

We've created this script for user conveniance, but the safest approach would be the method mentioned above. Before running the script, we would highly recommend creating a new Backup File containing your Users & Settings.

LetsEncrypt

Additionally, if you used LetsEncrypt to create an SSL cert, make sure to edit the new website.conf file to include your hostname, as the script will install a new website.conf file during the upgrade. The new website.conf file is located at these locations depending on your Cloud Provider.

Azure & Google Cloud:

/etc/nginx/sites-available/website.conf

AWS:

/etc/nginx/conf.d/website.conf

Check out the Troubleshooting section of the LetsEncrypt article to see what the website.conf file should look like for you.

Memory Settings

The script also adds a new StorageLink configuration file, which controls the Memory Settings for the StorageLink application. The conf file is located at:

/opt/swiftgw/swiftgateway-1.1.0.conf

By default, the configuration file is set to use the Memory Settings for a VM that has 8GB of RAM. If your VM has more or less than 8GB of RAM, make sure to edit your conf file to reflect your VM size.

You can use this article to find out the correct Memory values for your VM size.

Troubleshooting

If you run the script and you become stuck at the Please wait while StorageLink finishes setting up loading screen, contact us at support@thorntech.com and send us the most recent application log (Date is included in the name):

/opt/swiftgw/log/application-2024-05-1.log

Running the Script

SSH into the VM and run this command to elevate your privileges:

sudo su

If you have run the in-place upgrade script before, make sure you're in a different directory than where you previously ran it. You can run this command to create a new directory and move into it:

mkdir 110-upgrade
cd 110-upgrade

Next, run a wget command to download the script:

wget https://thorntech-products.s3.amazonaws.com/storagelink/1.001.00/in-place-upgrade-storagelink.sh

Give the script execute permissions:

chmod +x in-place-upgrade-storagelink.sh

Update the available packages since we're installing Java 17:

apt-get update

Finally, run the script:

./in-place-upgrade-storagelink.sh

When you refresh your Web Admin UI you should now see an updated UI and version at the bottom.

Script Contents

Here are the contents of the script for reference:

#!/bin/bash

#
# Preparation
#

# Show debug output, and halt on errors
set -xe

# Must run script as root, or else show usage
if [[ $(whoami) != "root" ]]; then
  echo "Usage: sudo $0"
  exit 1
fi

function extractPropValueFromSourceFile {
    local prefix="${1}"
    local str=`grep "${prefix}" ${2} 2>/dev/null`
    echo "${str#$prefix}" | xargs
}

# If on version 2, exit script as the command sftpgw version is only on version 2.x
command -v sftpgw version >/dev/null && exit

# Set target version
TARGET_VERSION="3.5.0"
TARGET_VERBOSE_VERSION="3.005.00"

# Set date
TODAY=$(date +"%m%d%Y")

APPLICATION_PROPERTIES="/opt/sftpgw/application.properties"

# Determine the cloud provider
AWS_DOMAIN=$(curl -s "http://169.254.169.254/latest/meta-data/services/domain")
AZURE_DOMAIN=$(curl --noproxy "*" -H 'Metadata: True' "http://169.254.169.254/metadata/instance/compute/azEnvironment?api-version=2019-06-01&format=text")
CLOUD_PROVIDER=gcp
[[ $AWS_DOMAIN   == "amazonaws.com"    ]] && CLOUD_PROVIDER=aws
[[ $AZURE_DOMAIN == "AzurePublicCloud" ]] && CLOUD_PROVIDER=azure

# Determine operating system and Nginx user
if getent passwd www-data > /dev/null 2>&1; then
  OS=ubuntu
  NGINX_USER=www-data
  NGINX_CONF_PATH="/etc/nginx/sites-available"
else
  OS=centos
  NGINX_USER=nginx
  NGINX_CONF_PATH="/etc/nginx/conf.d"
fi

if [ $OS == ubuntu ]; then
  apt-cache search openjdk-17-jre-headless
  DEBIAN_FRONTEND=noninteractive apt-get install openjdk-17-jre-headless -y -q
else
  yum install -y java-17-amazon-corretto
  /usr/sbin/update-alternatives --set java /usr/lib/jvm/java-17-amazon-corretto.x86_64/bin/java
fi

function generate_password() {
  local length=${1:-16}
  echo -n "$(
    head /dev/urandom | tr -dc A-Z0-9 | head -c $length
    echo ''
  )"
}

function set_jwt_secret_in_application_properties() {
  local jwt_secret=${1}
  echo "Remove any existing security.jwt.secret property"
  sudo sed -i.bak '/^security\.jwt\.secret=/d' ${APPLICATION_PROPERTIES}
  echo "Setting security.jwt.secret"
  echo -e "security.jwt.secret=$jwt_secret" | sudo tee -a ${APPLICATION_PROPERTIES}
}

function synchronize_jwt_secret() {
  local property1_key='jwt_secret'
  local property1_value=$OAUTH_JWT_SECRET

  read -r -d '' SQL_COMMAND <<EOF
    WITH old AS (
      SELECT
        key,
        value
      FROM properties
      WHERE application = 'sftpgateway'
        AND profile = ''
        AND label = ''
        AND (key = '$property1_key')
    ),
    new AS (
     INSERT INTO properties (application, profile, label, key, value)
       VALUES ('sftpgateway', '', '', '$property1_key', '${property1_value}')
       ON CONFLICT DO NOTHING
       RETURNING key, value
    )
    SELECT
      key,
      value
    FROM new
    UNION ALL
    SELECT
      key,
      value
    FROM old
    order by key;
EOF

  RESULTS=$(sudo -i -u postgres psql --command="$SQL_COMMAND" -Xt -d sftpgw)

  SAVED_OAUTH_JWT_SECRET=$(echo "$RESULTS" | cut -d'|' -f 2 | xargs)
  if [[ "$SAVED_OAUTH_JWT_SECRET" != "$OAUTH_JWT_SECRET" ]]; then
    echo "Existing JWT Secret found in database"
    set_jwt_secret_in_application_properties "$SAVED_OAUTH_JWT_SECRET"
    sudo systemctl restart sftpgw-admin-api
  fi
}

## Make proper database changes

sudo chsh -s /bin/bash postgres || echo ""
sudo -i -u postgres psql -d sftpgw -c "update databasechangelog set md5sum = null;"
sudo -i -u postgres psql -d sftpgw -c "TRUNCATE TABLE oauth_access_token;"

OAUTH_JWT_SECRET=$(generate_password 128)
set_jwt_secret_in_application_properties "$OAUTH_JWT_SECRET"

if ! grep -e "server.forward-headers-strategy=" ${APPLICATION_PROPERTIES}; then
  echo -e "server.forward-headers-strategy=framework" | sudo tee -a ${APPLICATION_PROPERTIES}
fi

if ! grep -e "security.redirect.base-path=" ${APPLICATION_PROPERTIES}; then
  echo -e "security.redirect.base-path=backend/" | sudo tee -a ${APPLICATION_PROPERTIES}
fi

#
# Install SFTP Gateway files
#

# Download public resources
wget https://thorntech-products.s3.amazonaws.com/sftpgateway/3.005.00/assets.zip
unzip assets.zip

# Install Java files

# jar
cd assets
chmod +x sftpgateway-admin-api-${TARGET_VERSION}.jar
chown sftpgw:sftpgw sftpgateway-admin-api-${TARGET_VERSION}.jar
mv sftpgateway-admin-api-${TARGET_VERSION}.jar /opt/sftpgw/

# conf
chown sftpgw:sftpgw sftpgateway-admin-api-${TARGET_VERSION}.conf
mv sftpgateway-admin-api-${TARGET_VERSION}.conf /opt/sftpgw/

chown ${NGINX_USER}:${NGINX_USER} website.conf
mv ${NGINX_CONF_PATH}/website.conf ${NGINX_CONF_PATH}/website.conf-${TODAY}
mv website.conf ${NGINX_CONF_PATH}

# Install website files
mv admin-ui.tar.gz /usr/share/nginx
cd /usr/share/nginx
mv admin-ui admin-ui-${TODAY}
tar xzvpf admin-ui.tar.gz && rm -f $_
chown -R ${NGINX_USER}:${NGINX_USER} admin-ui

# Populate the application properties file
CLIENT_ID=$(extractPropValueFromSourceFile "security.client-id=" ${APPLICATION_PROPERTIES})
CLIENT_SECRET=$(extractPropValueFromSourceFile "security.client-secret=" ${APPLICATION_PROPERTIES})
(
cat <<EOF
window._env_ = {
     "clientid": "$CLIENT_ID",
     "clientsecret": "$CLIENT_SECRET",
     "cloudProvider": "$CLOUD_PROVIDER",
     "version": "$TARGET_VERSION"
};
EOF
) | sudo tee /usr/share/nginx/admin-ui/webconfig.js
cd admin-ui
chown -R ${NGINX_USER}:${NGINX_USER} webconfig.js

# Update the version
service sftpgw-admin-api stop
cd /etc/systemd/system/
cp -a sftpgw-admin-api.service sftpgw-admin-api.service-${TODAY}
sed -i "s/sftpgateway-admin-api-.*.jar/sftpgateway-admin-api-${TARGET_VERSION}.jar/" sftpgw-admin-api.service
sed -i "s/e.activeVersion/\"${TARGET_VERSION}\"/" /usr/share/nginx/admin-ui/static/js/main.*.chunk.js
sed -i "s/3.*/${TARGET_VERSION}/" /etc/profile.d/login-info.sh
sed -i '15s/.*/TimeoutSec=900/' /etc/systemd/system/sftpgw-admin-api.service
sed -i 's/-XX:MaxPermSize=.*m //' /opt/sftpgw/sftpgateway-admin-api-*.conf

# Restart Nginx
nginx -t && service nginx restart

# Restart Java
systemctl daemon-reload
service sftpgw-admin-api start

synchronize_jwt_secret
sudo chsh -s /sbin/nologin postgres || echo ""