SFTP Gateway Support

3.0

HA Compute-Only Terraform Template

Overview

The previous KB article covers how to deploy SFTP Gateway as an HA stack using Terraform. This article builds off that idea, separating the Terraform template into 3 different stacks:

  • Compute
  • IAM
  • DB

In Enterprise environments, customers may want to deploy only the compute resources and point to an existing database. The Terraform templates in this article can be used as a reference to demonstrate how the compute and database layers can be separated. It also shows you which parameters to pass between the Terraform stacks in order to wire them together.

Folder hierarchy

In this article, you will end up creating the following files and folders:

/01_db/
/01_db/db.tf
/01_db/terraform.tfvars
/02_iam/
/02_iam/iam.tf
/02_iam/terraform.tfvars
/03_compute/
/03_compute/compute.tf
/03_compute/terraform.tfvars

Each of the following sections will walk you through this process.

The end of the article has a section for each of the files' contents.

Deploy the DB layer

SFTP Gateway, when deployed in HA, depends on an external database service. You can have your own DB team provision this for you. In this article, an example database Terraform template is provided so you can set up a working example for reference.

Create the following folder:

/01_db/

Within this folder, create the following files:

/01_db/db.tf
/01_db/terraform.tfvars

Populate these files with the examples at the end of this article.

For the terraform.tfvars file, replace the project with your actual GCP project name. Also, update the values for region and zone.

Finally, deploy the Terraform template:

terraform init
terraform plan
terraform apply

Make a note of the database name. For example, if the stack name is:

sftpgw-db-only

The database name looks like this:

sftpgw-db-only-db-instance

Deploy the IAM layer

SFTP Gateway connects to the database via an IAM service account. In this section, you will deploy this IAM service account in a separate Terraform template.

Create the following folder:

/02_iam/

Within this folder, create the following files:

/02_iam/iam.tf
/02_iam/terraform.tfvars

Populate the file contents with the examples at the end of this article.

For the terraform.tfvars file, replace the project with your actual GCP project name. Also, update the values for region and zone.

Update the db_instance_name property with the actual database name. If you used the stack name from our example, your database would have the name of sftpgw-db-only-db-instance.

The reason why you need to pass in the database name is because the IAM Terraform template takes the IAM service account, and inserts it as a database user.

Finally, deploy the Terraform template:

terraform init
terraform plan
terraform apply

Set up the database

In this section, you will perform some manual post-configuration steps on the database. Normally, these steps are performed by SFTP Gateway on first launch. But in Enterprise environments, you may not necessarily want SFTP Gateway to have the ability to perform these functions. This is why you need to perform these steps manually.

Change the postgres password

In order to perform database commands, you need the password for the postgres database user.

In the GCP console, go to your Cloud SQL database.

In the left pane, navigate to Users

Look for the user named postgres

Click on the 3 dots to expand the context menu, and then choose Change Password

Set the password for the postgres SFTP user

Connect to the database

In the next section, you will be performing database commands.

There are different ways to connect to the database, and you may already have a bastion host set up to connect to the database private IP. If this is the case, skip ahead to the next section.

If you're just testing this out and want to get this working, you can enable a public IP on the database.

Go to the Cloud SQL database Overview section

Click Edit at the top

Scroll down to the Customize your instance section

Expand the Connections section

Check the box next to Public IP

Scroll to the bottom and click Save

Wait for the edit operation you started to complete

Once you have a public IP, you can connect using Cloud Shell.

Go to the Cloud SQL database Overview section

Go to the Connect to this instance section

Toward the bottom of that section, click OPEN CLOUD SHELL

If prompted, click Authorize

Cloud Shell will open with the following pre-populated command:

gcloud sql connect sftpgw-db-only-db-instance --user=postgres --quiet

Hit Enter to connect to the database

Paste in the database password you set in the previous section

If successful, you should get a postgres=> command prompt.

Run the manual database commands

Once you are connected to the database, run the following command:

ALTER USER "sftpgw-iam-only-iam@sftp-gateway.iam" CREATEDB;

This command grants the IAM service account the ability to create databases. Make sure you replace the IAM service account with the actual account you created in the prior sections.

Note: The IAM service account should appear as sftpgw-iam-only-iam@sftp-gateway.iam rather than sftpgw-iam-only-iam@sftp-gateway.iam.gserviceaccount.com. In other words, the suffix .gserviceaccount.com should be truncated.

Next, create the database:

CREATE DATABASE sftpgw;

Next, use the database:

\c sftpgw;

Enter in the database password if prompted to do so.

Once you are in the sftpgw database, create the ltree extension:

create extension ltree;

That is it for the database commands. If you added a public IP to the database, you can remove it now.

Deploy the compute layer

Now that the database and IAM layers have been deployed, you can deploy the compute layer.

The compute layer is fairly similar to the Terraform HA template, with one notable difference:

export DO_CREATE_DB="no"

In the userdata, we pass in a variable indicating that we do not want to create the database. This way, we can limit the VMs to the permissions given to the IAM service account.

Create the following folder:

/03_compute/

Within this folder, create the following files:

/03_compute/compute.tf
/03_compute/terraform.tfvars

Populate the file contents with the examples at the end of this article.

For the terraform.tfvars file, replace the project with your actual GCP project name. Also, update the values for region and zone.

For the web_admin_ip_address property, set this to your current public IP, followed by /32. An easy to remember website to check your IP is ipchicken.com.

For the db_host property, replace this with Connection name of your database. For example: sftp-gateway:us-central1:sftpgw-db-only-db-instance.

For the image_path, this points to SFTP Gateway Professional. The image ID will vary depending on the exact version you want to deploy.

For the google_service_account_email property, set this to the full email address of the IAM service account created in a prior section. For example:

sftpgw-iam-only-iam@sftp-gateway.iam.gserviceaccount.com

Finally, deploy the Terraform template:

terraform init
terraform plan
terraform apply

This Terraform template will deploy two VMs of SFTP Gateway behind a load balancer.

Troubleshooting

At the time of this writing, the SFTP Gateway VMs are able to deploy. However, the Nginx service on each instance needs to be restarted:

service nginx restart

Afterwards, the instances should behave normally.

File contents

Refer to this section for the Terraform file contents.

/01_db/db.tf

terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
      version = "5.2.0"
    }
  }
}

provider "google" {
  credentials = var.credentials == null ? null : file(var.credentials)
  project = var.project
  region  = var.region
  zone    = var.zone
}

variable "credentials" {
  type = string
  description = "Name of your Service Key"
  default = null
}

variable "project" {
  type = string
  description = "Name of your Google Cloud project"
}

variable "stack_name" {
  type = string
  description = "Name of this Terraform stack, which acts as a prefix for resources"
}

variable "region" {
  type = string
  description = "Name of your Google Cloud region"
  default = "us-central1"
}

variable "zone" {
  type = string
  description = "Name of your region zone"
  default = "us-central1-c"
}

resource "random_password" "db_password" {
  length           = 20
  special          = true
  override_special = "*+?&^!@"
}

# DB

resource "google_sql_database" "database" {
  name     = "${var.stack_name}-sftpgw"
  instance = google_sql_database_instance.instance.name
  charset = "utf8"
}

resource "google_sql_database_instance" "instance" {
  name             = "${var.stack_name}-db-instance"
  database_version = "POSTGRES_13"
  root_password = "${random_password.db_password.result}"
  settings {
    tier = "db-custom-4-16384"
    disk_size = 20
    disk_type = "PD_SSD"
    availability_type = "REGIONAL"
    disk_autoresize = true
    activation_policy = "ALWAYS"
    maintenance_window {
      hour = 0
      day  = 7
    }
    backup_configuration {
      enabled = true
      start_time = "00:00"
    }
    ip_configuration {
      ipv4_enabled = false
      private_network = "https://www.googleapis.com/compute/v1/projects/${var.project}/global/networks/default"
    }
    database_flags {
      name = "cloudsql.iam_authentication"
      value = "on"
    }
  }
}

output "db_instance_name" {
  value = "${google_sql_database_instance.instance.name}"
}

output "DB_HOST" {
  value = "${google_sql_database_instance.instance.connection_name}"
}

output "random_password" {
  value = random_password.db_password.result
  sensitive = true
}

/01_db/terraform.tfvars

stack_name = "sftpgw-db-only"
project = "sftp-gateway"
region = "us-central1"
zone = "us-central1-c"

/02_iam/iam.tf

terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
      version = "5.2.0"
    }
  }
}

provider "google" {
  credentials = var.credentials == null ? null : file(var.credentials)
  project = var.project
  region  = var.region
  zone    = var.zone
}

variable "credentials" {
  type = string
  description = "Name of your Service Key"
  default = null
}

variable "project" {
  type = string
  description = "Name of your Google Cloud project"
}

variable "stack_name" {
  type = string
  description = "Name of this Terraform stack, which acts as a prefix for resources"
}

variable "region" {
  type = string
  description = "Name of your Google Cloud region"
  default = "us-central1"
}

variable "zone" {
  type = string
  description = "Name of your region zone"
  default = "us-central1-c"
}

locals {
  truncated_email = trimsuffix(google_service_account.service_account_solution.email, ".gserviceaccount.com")
}

variable "db_instance_name" {
  type = string
  description = "DB instance name"
}

# IAM

resource "google_service_account" "service_account_solution" {
  account_id = "${var.stack_name}-iam"
  display_name = "SFTPGW Service Account Terraform"
  project = var.project
}

resource "google_project_iam_member" "role_service_account_sql_instance" {
  role = "roles/cloudsql.instanceUser"
  member = "serviceAccount:${google_service_account.service_account_solution.email}"
  project = var.project
}

resource "google_project_iam_member" "role_service_account_sql_client" {
  role = "roles/cloudsql.client"
  member = "serviceAccount:${google_service_account.service_account_solution.email}"
  project = var.project
}

resource "google_project_iam_member" "role_service_account_logging" {
  role = "roles/logging.serviceAgent"
  member = "serviceAccount:${google_service_account.service_account_solution.email}"
  project = var.project
}

resource "google_project_iam_member" "role_service_account_storage" {
  role = "roles/storage.admin"
  member = "serviceAccount:${google_service_account.service_account_solution.email}"
  project = var.project
}

resource "google_project_iam_member" "role_service_account_metric_write" {
  role = "roles/monitoring.metricWriter"
  member = "serviceAccount:${google_service_account.service_account_solution.email}"
  project = var.project
}

resource "google_project_iam_member" "role_service_account_log_write" {
  role = "roles/logging.logWriter"
  member = "serviceAccount:${google_service_account.service_account_solution.email}"
  project = var.project
}

resource "google_project_iam_member" "role_service_account_log_view_accessor" {
  role = "roles/logging.viewAccessor"
  member = "serviceAccount:${google_service_account.service_account_solution.email}"
  project = var.project
}

resource "google_sql_user" "db-service-user" {
  name = local.truncated_email
  instance = var.db_instance_name
  type = "CLOUD_IAM_SERVICE_ACCOUNT"
}

/02_iam/terraform.tfvars

stack_name = "sftpgw-iam-only"
project = "sftp-gateway"
region = "us-central1"
zone = "us-central1-c"
db_instance_name = "sftpgw-db-only-db-instance"

/03_compute/compute.tf

terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
      version = "5.2.0"
    }
  }
}

provider "google" {
  credentials = var.credentials == null ? null : file(var.credentials)
  project = var.project
  region  = var.region
  zone    = var.zone
}

variable "credentials" {
  type = string
  description = "Name of your Service Key"
  default = null
}

variable "project" {
  type = string
  description = "Name of your Google Cloud project"
}

variable "stack_name" {
  type = string
  description = "Name of this Terraform stack, which acts as a prefix for resources"
}

variable "region" {
  type = string
  description = "Name of your Google Cloud region"
  default = "us-central1"
}

variable "zone" {
  type = string
  description = "Name of your region zone"
  default = "us-central1-c"
}

variable "image_path" {
  type = string
  description = "Path to the VM image"
}

variable "web_admin_ip_address" {
  type = string
  description = "IP address range of your workstation. Provides sysadmin access for SSH and web administration."
}

variable "machine_type" {
  type = string
  description = "Machine type of your VM"
  default = "e2-medium"
}

variable "google_service_account_email" {
  type = string
  description = "Google service account email"
}

variable "db_host" {
  type = string
}

# Compute

resource "google_compute_instance_template" "it" {
  name = "${var.stack_name}-instance-template"
  machine_type = var.machine_type
  network_interface {
    network = "default"
    access_config {}
  }
  service_account {
    email = var.google_service_account_email
    scopes = [
      "https://www.googleapis.com/auth/cloud.useraccounts.readonly",
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring.write",
      "https://www.googleapis.com/auth/cloud-platform"
    ]
  }
  disk {
    auto_delete = true
    type = "PERSISTENT"
    device_name = "boot"
    boot = true
    source_image = var.image_path
    disk_size_gb = 32
    disk_type = "pd-ssd"
  }
  can_ip_forward = false
  tags = [
    "${var.stack_name}-firewall-health-check",
    "${var.stack_name}-public-network",
    "${var.stack_name}-admin-network"
  ]
  metadata = {
    google-logging-enable = 1
    google-monitoring-enable = 1
    user-data = <<-EOT
#cloud-config
repo_update: true
repo_upgrade: all

write_files:
- content : |
    #!/bin/bash
    export CLOUD_PROVIDER=gcp
    export ARCHITECTURE=HA
    export DB_HOST=${var.db_host}
    export DO_CREATE_DB="no"
  path: /opt/sftpgw/launch_config.env
- path: /opt/sftpgw/application.properties
  append: true
  content: |
    features.first-connection.cloud-provider=gcp
    features.first-connection.use-instance-credentials=true

EOT
  }
}

resource "google_compute_region_instance_group_manager" "igm" {
  name = "${var.stack_name}-igm"
  region = var.region
  base_instance_name = var.stack_name
  version {
    instance_template = google_compute_instance_template.it.id
  }
  target_size = 2
  named_port {
    name = "http"
    port = 80
  }
  named_port {
    name = "https"
    port = 443
  }
  named_port {
    name = "ssh"
    port = 2222
  }
  named_port {
    name = "sftp"
    port = 22
  }
  depends_on = [
    google_compute_instance_template.it
  ]
}

resource "google_compute_firewall" "public-network" {
  name    = "${var.stack_name}-public-network"
  network = "default"
  target_tags = ["${var.stack_name}-public-network"]
  source_ranges = [
    "0.0.0.0/0"
  ]
  allow {
    protocol = "tcp"
    ports    = ["22"]
  }
}

resource "google_compute_firewall" "admin-network" {
  name    = "${var.stack_name}-admin-network"
  network = "default"
  target_tags = ["${var.stack_name}-admin-network"]
  source_ranges = [
    var.web_admin_ip_address,
    "35.235.240.0/20" // Google Cloud Portal SSH tool
  ]
  allow {
    protocol = "tcp"
    ports    = ["2222", "80", "443"]
  }
}

/03_compute/terraform.tfvars

stack_name = "sftpgw-compute-only"
project = "sftp-gateway"
region = "us-central1"
zone = "us-central1-c"
web_admin_ip_address = "50.219.57.90/32"
db_host = "sftp-gateway:us-central1:sftpgw-db-only-db-instance"
image_path = "https://www.googleapis.com/compute/v1/projects/thorn-technologies-public/global/images/sftpgw-pro-3-7-1-20250328002321"
google_service_account_email = "sftpgw-iam-only-iam@sftp-gateway.iam.gserviceaccount.com"
← PreviousNext →