CloudFormation: HA new network
Overview
SFTP Gateway version 3.3.0
has a hidden high availability (HA) feature.
Normally, you would deploy an HA CloudFormation template from the AWS Marketplace (which will be available in our next version, 3.3.1
). But for the time being, you can use the instructions in this article to manually deploy an HA template.
This article shows you how to deploy SFTP Gateway version 3 with an HA configuration into a New VPC network.
CloudFormation deployment instructions
This section walks you through deploying the HA New Network CloudFormation template.
First, create a CloudFormation template file, using the yaml code found at the bottom of this article.
Then, in the AWS console, go to the CloudFormation service.
Go to Create stack > With new resources (standard).
On the Create stack page, select Upload a template file.
Click Choose file, and choose the yaml file you created earlier.
On the Specify stack details page, enter the following information:
- Stack name: The name of your CloudFormation stack.
- Bucket Access: The default is
restricted
, which limits access to S3 buckets following a naming convention. Chooseopen
to allow access to all S3 buckets. - EC2 Type: Defaults to
t3.medium
. We recommend usingm5.large
for production workloads. - Disk Volume Size: Defaults to
32 GB
. We use S3 as the file system, so this should be enough to accommodate the OS and rotated logs. - Key Pair: Choose a
Key Pair
that you have access to. - Desired Capacity: Defaults to
2
. This determines how many instances you want to run. - VPC IP Range: Select a Class C IP range for your VPC. Defaults to
192.168.1.0/24
. - Input CIDR: Enter a CIDR range (e.g.
1.2.3.4/32
) that represents the IP address your workstation. This opens an ingress rule for sysadmin use. - SFTP Input CIDR: Enter a CIDR range for the SFTP service. Defaults to
0.0.0.0/0
. - DB Class: This is the database service instance class. Defaults to
db.t3.micro
, since SFTP Gateway is not database-heavy.
When finished on this page, click Next.
On the Configure stack options page, leave the defaults, and click Next.
On the Review page, scroll to the bottom and check the box to acknowledge that CloudFormation is going to create IAM resources. This is because the template creates an IAM role for the EC2 instance for accessing S3.
Finally, click Create Stack.
Post launch instructions
When the CloudFormation stack is created, go to the Outputs tab.
For the Hostname field, there will be a link to the Network Load Balancer's DNS hostname. Click this link, and it will take you to the web admin portal.
Then, configure SFTP Gateway as you normally would, by creating the first web admin account in the First Launch Experience.
HA New Network CloudFormation template
Below is the HA New Network CloudFormation template. Copy the contents of this code block, and create a yaml file on your workstation.
AWSTemplateFormatVersion: 2010-09-09
Mappings:
RegionMap: # 3.3.0
us-east-1:
AMI: ami-06d14857eebdce422
us-gov-east-1:
AMI: ami-007b1304a84996436
us-gov-west-1:
AMI: ami-0b452e9bf23151cf5
us-east-2:
AMI: ami-0738ab03116dab4d4
us-west-1:
AMI: ami-0eb385ee40d0b81be
us-west-2:
AMI: ami-0365945dcbc685b14
ca-central-1:
AMI: ami-0bc63dafabda05e39
eu-central-1:
AMI: ami-0471ff0fb21150381
eu-west-1:
AMI: ami-009eb82c412a75ef7
eu-west-2:
AMI: ami-090b56704420a1870
eu-west-3:
AMI: ami-0daa16f6dd279d91c
eu-north-1:
AMI: ami-03f465518ac29ca01
eu-south-1:
AMI: ami-07a46eaab793b51f4
ap-east-1:
AMI: ami-0656e09f139b603d8
ap-southeast-1:
AMI: ami-0878a6adfbf75f655
ap-southeast-2:
AMI: ami-057bd057e564378e0
ap-south-1:
AMI: ami-07191a62016ec4615
ap-northeast-1:
AMI: ami-0bb21be71d570676b
ap-northeast-2:
AMI: ami-047d9ff1fc7550df7
me-south-1:
AMI: ami-0c72989ee662ed182
Conditions:
S3IsOpen: !Equals
- "open"
- !Ref BucketAccess
S3IsRestricted: !Equals
- "restricted"
- !Ref BucketAccess
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: Bucket
Parameters:
- BucketAccess
- Label:
default: Instance and storage options
Parameters:
- EC2Type
- DiskVolumeSize
- KeyPair
- DesiredCapacity
- Label:
default: Network
Parameters:
- VPCIPRange
- Label:
default: Access and authentication
Parameters:
- InputCIDR
- SFTPInputCIDR
Parameters:
EC2Type:
Description: SFTP Gateway instance type. You can use a t3.medium for testing, but m5.large is recommended for Production.
Type: String
Default: t3.medium
AllowedValues:
- t2.micro
- t2.small
- t2.medium
- t2.large
- t2.xlarge
- t2.2xlarge
- t3.micro
- t3.small
- t3.medium
- t3.large
- t3.xlarge
- t3.2xlarge
- m3.medium
- m3.large
- m3.xlarge
- m3.2xlarge
- m4.large
- m4.xlarge
- m4.2xlarge
- m4.4xlarge
- m4.10xlarge
- m4.16xlarge
- m5.large
- m5.xlarge
- m5.2xlarge
- m5.4xlarge
- m5.12xlarge
- m5.24xlarge
- c3.large
- c3.xlarge
- c3.2xlarge
- c3.4xlarge
- c3.8xlarge
- c4.large
- c4.xlarge
- c4.2xlarge
- c4.4xlarge
- c4.8xlarge
- c5.large
- c5.xlarge
- c5.2xlarge
- c5.4xlarge
- c5.9xlarge
- c5.18xlarge
- f1.2xlarge
- f1.16xlarge
- g2.2xlarge
- g2.8xlarge
- g3.4xlarge
- g3.8xlarge
- g3.16xlarge
- p2.xlarge
- p2.8xlarge
- p2.16xlarge
- p3.2xlarge
- p3.8xlarge
- p3.16xlarge
- cr1.8xlarge
- r3.large
- r3.xlarge
- r3.2xlarge
- r3.4xlarge
- r3.8xlarge
- r4.large
- r4.xlarge
- r4.2xlarge
- r4.4xlarge
- r4.8xlarge
- r4.16xlarge
- r5.large
- r5.xlarge
- r5.2xlarge
- r5.4xlarge
- r5.12xlarge
- r5.24xlarge
- r5d.large
- r5d.xlarge
- r5d.2xlarge
- r5d.4xlarge
- r5d.12xlarge
- r5d.24xlarge
- z1d.large
- z1d.xlarge
- z1d.2xlarge
- z1d.3xlarge
- z1d.6xlarge
- z1d.12xlarge
- x1.16xlarge
- x1.32xlarge
- x1e.xlarge
- x1e.2xlarge
- x1e.4xlarge
- x1e.8xlarge
- x1e.16xlarge
- x1e.32xlarge
- i2.xlarge
- i2.2xlarge
- i2.4xlarge
- i2.8xlarge
- i3.large
- i3.xlarge
- i3.2xlarge
- i3.4xlarge
- i3.8xlarge
- i3.16xlarge
- h1.2xlarge
- h1.4xlarge
- h1.8xlarge
- h1.16xlarge
- hs1.8xlarge
- d2.xlarge
- d2.2xlarge
- d2.4xlarge
- d2.8xlarge
- a1.medium
- a1.large
- a1.xlarge
- a1.2xlarge
- a1.4xlarge
- c5n.large
- c5n.xlarge
- c5n.2xlarge
- c5n.4xlarge
- c5n.9xlarge
- c5n.18xlarge
VPCIPRange:
Description: Class C IP range
AllowedValues:
- 192.168.1.0/24
- 192.168.2.0/24
- 192.168.3.0/24
- 192.168.4.0/24
- 192.168.5.0/24
- 192.168.6.0/24
- 192.168.7.0/24
- 192.168.8.0/24
- 192.168.9.0/24
- 192.168.10.0/24
- 192.168.11.0/24
- 192.168.12.0/24
- 192.168.13.0/24
- 192.168.14.0/24
- 192.168.15.0/24
- 192.168.16.0/24
- 192.168.17.0/24
- 192.168.18.0/24
- 192.168.19.0/24
- 192.168.20.0/24
Default: 192.168.1.0/24
Type: String
DiskVolumeSize:
Default: 32
Description: Disk volume size in GB. Must be at least 32.
ConstraintDescription: Must be a number greater or equal to 32
MinValue: 32
Type: Number
InputCIDR:
Description: Public IP address range for SSH and web access. Use a CIDR range to restrict access. To get your local machine's IP, see http://checkip.dyndns.org/. (Remember to append /32 for a single IP e.g. 12.34.56.78/32) For security reasons, do not use 0.0.0.0/0.
AllowedPattern: ([1-9]\d{0,2})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/([1-9]\d{0,1})
ConstraintDescription: Must be a valid IP CIDR range in the form of x.x.x.x/x. Do not use 0.0.0.0/0.
Type: String
DBClass:
AllowedValues:
- db.t3.micro
- db.t3.small
- db.t3.medium
- db.m5.large
ConstraintDescription: Must select a valid database instance type.
Default: db.t3.micro
Description: DB instance type
Type: String
DesiredCapacity:
Description: Desired capacity
Type: Number
Default: 2
SFTPInputCIDR:
Default: 0.0.0.0/0
Description: Public IP address range for SFTP access. Defaults to being open to the public (0.0.0.0/0), since the web admin portal lets you restrict IP ranges on a per-user basis.
AllowedPattern: (\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,3})
ConstraintDescription: Must be a valid IP CIDR range in the form of x.x.x.x/x.
Type: String
KeyPair:
Description: Make sure you have access to this EC2 key pair. Otherwise, create a new key pair before proceeding.
Type: AWS::EC2::KeyPair::KeyName
ConstraintDescription: Existing EC2 KeyPair.
BucketAccess:
Description: Choose "open" to allow full S3 access to support multiple buckets. Or "restrict" S3 permissions to our default bucket naming convention "sftpgw-<instance-id>".
Default: restricted
Type: String
AllowedValues:
- open
- restricted
Resources:
InternetGateway:
Type: AWS::EC2::InternetGateway
AttachGateway:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
InternetGatewayId: !Ref InternetGateway
VpcId: !Ref VPC
PublicRouteAssociationA:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PublicRouteTable
SubnetId: !Ref PublicSubnetA
PublicRouteAssociationB:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PublicRouteTable
SubnetId: !Ref PublicSubnetB
PublicRouteTable:
DependsOn: AttachGateway
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
PublicDefaultRoute:
Type: AWS::EC2::Route
Properties:
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
RouteTableId: !Ref PublicRouteTable
PublicSubnetA:
DependsOn:
- AttachGateway
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: !Select
- 0
- !GetAZs
Ref: AWS::Region
CidrBlock: !Select [ 0, !Cidr [ !Ref VPCIPRange, 16, 4 ]]
MapPublicIpOnLaunch: true
VpcId: !Ref VPC
PublicSubnetB:
DependsOn:
- AttachGateway
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: !Select
- 1
- !GetAZs
Ref: AWS::Region
CidrBlock: !Select [ 1, !Cidr [ !Ref VPCIPRange, 16, 4 ]]
MapPublicIpOnLaunch: true
VpcId: !Ref VPC
PrivateRouteAssociationApp0:
Properties:
RouteTableId:
Ref: PrivateRoutingTable
SubnetId:
Ref: PrivateSubnetA
Type: AWS::EC2::SubnetRouteTableAssociation
PrivateRouteAssociationApp1:
Properties:
RouteTableId:
Ref: PrivateRoutingTable
SubnetId:
Ref: PrivateSubnetB
Type: AWS::EC2::SubnetRouteTableAssociation
PrivateRoutingTable:
Properties:
VpcId:
Ref: VPC
Type: AWS::EC2::RouteTable
PrivateSubnetA:
DependsOn:
- AttachGateway
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: !Select
- 0
- !GetAZs
Ref: AWS::Region
CidrBlock: !Select [ 2, !Cidr [ !Ref VPCIPRange, 16, 4 ] ]
MapPublicIpOnLaunch: false
VpcId: !Ref VPC
PrivateSubnetB:
DependsOn:
- AttachGateway
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: !Select
- 1
- !GetAZs
Ref: AWS::Region
CidrBlock: !Select [ 3, !Cidr [ !Ref VPCIPRange, 16, 4 ] ]
MapPublicIpOnLaunch: false
VpcId: !Ref VPC
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref VPCIPRange
EnableDnsHostnames: true
DBSecretStore:
Type: AWS::SecretsManager::Secret
Description: "Dynamically generated secret password for SFTP Gateway database access"
Properties:
GenerateSecretString:
SecretStringTemplate: '{"username": "sftpgw"}'
GenerateStringKey: "password"
PasswordLength: 30
ExcludePunctuation: true
IncludeSpace: false
Tags:
- Key: "Name"
Value: "SFTP Gateway DB Secret"
DBSecretAttachment:
Type: AWS::SecretsManager::SecretTargetAttachment
Properties:
SecretId: !Ref DBSecretStore
TargetId: !Ref SftpgwDB
TargetType: AWS::RDS::DBInstance
SftpgwDB:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceIdentifier: !Sub sftpgw-${AWS::StackName}
DBName: 'sftpgw'
DBInstanceClass: !Ref DBClass
DBSubnetGroupName: !Ref SftpgwDBSubnetGroup
AllocatedStorage: '50'
Engine: postgres
EngineVersion: '13.5'
MasterUsername: !Sub "{{resolve:secretsmanager:${DBSecretStore}::username}}"
MasterUserPassword: !Sub "{{resolve:secretsmanager:${DBSecretStore}::password}}"
MultiAZ: true
AutoMinorVersionUpgrade: true
StorageEncrypted: true
PubliclyAccessible: false
StorageType: gp2
VPCSecurityGroups:
- !Ref RDSSecurityGroup
Tags:
- Key: Name
Value: SFTP Gateway Database
SftpgwDBSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: Subnets available for the RDS DB Instance
SubnetIds:
- !Ref PrivateSubnetA
- !Ref PrivateSubnetB
RDSSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Security group for RDS DB Instance.
VpcId: !Ref VPC
DBSecurityGroupIngress:
Type: AWS::EC2::SecurityGroupIngress
Properties:
IpProtocol: tcp
FromPort: 5432
ToPort: 5432
SourceSecurityGroupId: !Ref SFTPGatewaySG
GroupId: !Ref RDSSecurityGroup
IPAddress:
Properties:
Domain: vpc
Type: AWS::EC2::EIP
NetworkLoadBalancer:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Type: network
LoadBalancerAttributes:
- Key: load_balancing.cross_zone.enabled
Value: "true"
SubnetMappings:
- AllocationId: !GetAtt IPAddress.AllocationId
SubnetId: !Ref PublicSubnetA
- SubnetId: !Ref PublicSubnetB
NLBListener22:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
DefaultActions:
- TargetGroupArn: !Ref NLBTargetGroup22
Type: forward
LoadBalancerArn: !Ref NetworkLoadBalancer
Port: 22
Protocol: TCP
NLBListener2222:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
DefaultActions:
- TargetGroupArn: !Ref NLBTargetGroup2222
Type: forward
LoadBalancerArn: !Ref NetworkLoadBalancer
Port: 2222
Protocol: TCP
NLBListener80:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
DefaultActions:
- TargetGroupArn: !Ref NLBTargetGroup80
Type: forward
LoadBalancerArn: !Ref NetworkLoadBalancer
Port: 80
Protocol: TCP
NLBListener443:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
DefaultActions:
- TargetGroupArn: !Ref NLBTargetGroup443
Type: forward
LoadBalancerArn: !Ref NetworkLoadBalancer
Port: 443
Protocol: TCP
NLBTargetGroup22:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Port: 22
Protocol: TCP
VpcId: !Ref VPC
TargetGroupAttributes:
- Key: "preserve_client_ip.enabled"
Value: true
HealthCheckEnabled: true
HealthCheckPort: 443
HealthCheckProtocol: HTTPS
NLBTargetGroup2222:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Port: 2222
Protocol: TCP
VpcId: !Ref VPC
HealthCheckEnabled: true
HealthCheckPort: 443
HealthCheckProtocol: HTTPS
NLBTargetGroup80:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Port: 80
Protocol: TCP
VpcId: !Ref VPC
NLBTargetGroup443:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Port: 443
Protocol: TCP
VpcId: !Ref VPC
AutoScalingGroup:
DependsOn:
- RolePolicies
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
DesiredCapacity: !Ref DesiredCapacity
LaunchConfigurationName: !Ref LaunchConfiguration
MaxSize: "10"
MinSize: "2"
TargetGroupARNs:
- !Ref NLBTargetGroup22
- !Ref NLBTargetGroup80
- !Ref NLBTargetGroup443
- !Ref NLBTargetGroup2222
VPCZoneIdentifier:
- !Ref PublicSubnetA
- !Ref PublicSubnetB
Tags:
- Key: Name
Value: SFTPGateway
PropagateAtLaunch: true
CreationPolicy:
ResourceSignal:
Count: !Ref DesiredCapacity
Timeout: PT15M
LaunchConfiguration:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
AssociatePublicIpAddress: true
IamInstanceProfile: !Ref EC2InstanceProfile
ImageId: !FindInMap
- RegionMap
- !Ref AWS::Region
- AMI
KeyName: !Ref KeyPair
InstanceType: !Ref EC2Type
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
VolumeSize: !Ref DiskVolumeSize
VolumeType: gp2
Encrypted: 'true'
SecurityGroups:
- !Ref SFTPGatewaySG
UserData:
Fn::Base64: !Sub |
#cloud-config
repo_update: true
repo_upgrade: all
write_files:
- content : |
#!/bin/bash
export CLOUD_PROVIDER=aws
export ARCHITECTURE=HA
export LOG_GROUP_NAME=${LogGroup}
export SECRET_ID=${DBSecretStore}
export DB_HOST=${SftpgwDB.Endpoint.Address}
path: /opt/sftpgw/launch_config.env
runcmd:
- /opt/aws/bin/cfn-init --stack ${AWS::StackName} --resource LaunchConfiguration --region ${AWS::Region}
- /opt/aws/bin/cfn-signal -e 0 --stack ${AWS::StackName} --resource AutoScalingGroup --region ${AWS::Region}
LogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub sftpgw-${AWS::StackName}
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles:
- !If [S3IsRestricted, !Ref RestrictedSFTPGWRole, !Ref OpenSFTPGWRole]
RestrictedSFTPGWRole:
Type: AWS::IAM::Role
Condition: S3IsRestricted
Properties:
Path: /
ManagedPolicyArns:
- !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonEC2RoleforSSM
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: ec2.amazonaws.com
Action: sts:AssumeRole
OpenSFTPGWRole:
Type: AWS::IAM::Role
Condition: S3IsOpen
Properties:
Path: /
ManagedPolicyArns:
- !Sub arn:${AWS::Partition}:iam::aws:policy/AmazonS3FullAccess
- !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonEC2RoleforSSM
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: ec2.amazonaws.com
Action: sts:AssumeRole
RolePolicies:
Type: AWS::IAM::Policy
Properties:
PolicyName: SFTPGatewayPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: 's3:*'
Resource: !Sub 'arn:${AWS::Partition}:s3:::sftpgw-i-*'
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:PutLogEvents
- logs:DescribeLogStreams
- logs:CreateLogGroup
Resource: '*'
- Effect: Allow
Action:
- ec2:DescribeAvailabilityZones
- ec2:DescribeInstances
- ec2:DescribeTags
- s3:ListAllMyBuckets
Resource: '*'
- Effect: Allow
Action:
- cloudformation:DescribeStacks
- cloudformation:ListStackResources
Resource: '*'
- Effect: Allow
Action:
- secretsmanager:GetResourcePolicy
- secretsmanager:GetSecretValue
- secretsmanager:DescribeSecret
- secretsmanager:ListSecretVersionIds
Resource:
- !Ref DBSecretStore
- Effect: Allow
Action:
- secretsmanager:ListSecrets
Resource: '*'
- Effect: Allow
Action:
- secretsmanager:GetResourcePolicy
- secretsmanager:GetSecretValue
- secretsmanager:DescribeSecret
- secretsmanager:ListSecretVersionIds
Resource:
- !Ref DBSecretStore
Roles:
- !If [S3IsRestricted, !Ref RestrictedSFTPGWRole, !Ref OpenSFTPGWRole]
SFTPGatewaySG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: SFTPGateway Security Group
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: !Ref SFTPInputCIDR
- IpProtocol: tcp
FromPort: 2222
ToPort: 2222
CidrIp: !Ref InputCIDR
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: !Ref InputCIDR
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: !Ref InputCIDR
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: !Ref VPCIPRange
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: !Ref VPCIPRange
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: !Ref VPCIPRange
SecurityGroupEgress:
- IpProtocol: tcp
FromPort: 0
ToPort: 65535
CidrIp: 0.0.0.0/0
- IpProtocol: udp
FromPort: 0
ToPort: 65535
CidrIp: 0.0.0.0/0
Outputs:
Hostname:
Value: !GetAtt NetworkLoadBalancer.DNSName
Description: Network Load Balancer DNS
CloudWatchLogs:
Value: !Ref LogGroup
Description: CloudWatch logs