SFTP Gateway 2.0 CloudFormation Timeout
Note: This page applies to SFTP Gateway version 2.x. Visit Here for documentation on version 3.x.
Overview
When spinning up the CloudFormation template for SFTP Gateway, it fails and
you get a ROLLBACK_COMPLETE. In the event history, you get the error:
ROLLBACK_IN_PROGRESS
Stack creation time exceeded the specified timeout. Rollback requested by user.
This happens because SFTP Gateway needs outbound internet access during its bootstrap process. For example, it needs to contact a public S3 endpoint to create the default S3 bucket. It also needs to contact a public resource hosted on S3 to determine the latest version.
SFTP Gateway could be hanging while trying to reach these endpoints. The CloudFormation template,
via the cfn-signal command, gives the EC2 instance 15 minutes before declaring a timeout and rolling
back the entire stack.
Troubleshooting approaches
To fix this issue, you need to figure out what is causing the SFTP Gateway instance to hang.
NAT Gateway
If you deploy SFTP Gateway into a private subnet, consider adding a NAT Gateway. Doing so provides outboud internet access.
Check your DHCP options
In an AD environment, you might have configured DHCP options on the VPC. These DHCP options point
any new server to your AD DNS servers. If you don't have the authority within your organization
to disable these DHCP options, you can bypass them by modifying the server settings.
1.) SSH into the EC2 instance
2.) Run nslookup google.com, which should fail
3.) Edit /etc/resolv.conf. You want it to look like this:
search ec2.internal
nameserver 172.31.0.2
You want to make sure that search ec2.internal appears (which may have been removed by the DHCP Option set).
Also, you want the nameserver IP address to end in .2. So if your private subnet is 192.168.0.0/24, you want the nameserver to
be 192.168.0.2. This is because AWS has a convention of assigning its internal DNS server the second (.2) IP address
of the subnet.
4.) Run nslookup google.com. It should work this time.
5.) Make the /etc/resolv.conf file immutable:
chattr +i /etc/resolv.conf
This prevents the DHCP Options set from reverting your change.
(6.) Force a DHCP renew by running the command:
dhclient