Uploading Very Large Files (Increasing the Maximum Upload Size)

TLDR - Quick Summary

What: Raise SFTP Gateway's maximum single-file upload size when your cloud connection writes to Amazon S3.

Why a limit exists: S3 multipart uploads allow at most 10,000 parts, so part size × 10,000 = the largest single object. The default part size (8 MB) caps a single file at ~80 GB.

Symptom: a file larger than the maximum fails partway through, and most SFTP clients retry from the beginning — so it looks like the transfer "keeps restarting."

Fix: raise features.file-system.aws-s3.max-simple-upload-size-bytes (the part size). Larger parts use more heap, so give the backend more memory.

Apply: container — set an environment variable and restart; EC2/marketplace — edit application.properties and run service sftpgw-admin-api restart.

Go further: SFTP Gateway for the Enterprise

Running secure file transfer at enterprise scale? Our free white paper covers high-availability SFTP Gateway architecture, security and compliance, and cloud integration across AWS, Azure, and Google Cloud.

👉 Download the SFTP Gateway white paper — free; a short registration unlocks the PDF.

Overview

SFTP Gateway uploads files to Amazon S3 with multipart uploads: each file is sent in parts, and S3 assembles the parts into one object. The part size is configurable, and S3 allows at most 10,000 parts per object, so part size × 10,000 = the largest single file SFTP Gateway can upload. At the default 8 MB part size, that is about 80 GB.

If a file is larger than that maximum, the upload fails partway through and most SFTP clients retry from the beginning, so it looks like the transfer "keeps restarting." Increasing the part size raises the ceiling.

This article extends the Application Properties reference. For broader throughput and memory tuning, see Performance Tuning SFTP Gateway.

Default S3 limit

Setting	Default	S3 part limit	Default max single file
features.file-system.aws-s3.max-simple-upload-size-bytes (bytes)	8 MB (8388608)	10,000 parts	about 80 GB

The same property is also the threshold below which SFTP Gateway sends a file as a single (non-multipart) upload, which is where the name comes from.

Choosing a part size

Maximum file size = part size × 10,000. So the part size you need is your largest expected file divided by 10,000.

Part size	Property value (bytes)	Maximum single file
8 MB (default)	8388608	about 80 GB
16 MB	16777216	about 160 GB
64 MB	67108864	about 640 GB
100 MB	104857600	about 1 TB

S3 allows a maximum of 5 GB per part and 5 TB per object, so 5 TB is the absolute ceiling for a single file.

How to change the setting

Container edition (Docker, Kubernetes, ECS)

Set the environment variable on the SFTP Gateway backend container and restart it. The value is in bytes.

Environment variable	Example value
FEATURES_FILESYSTEM_AWSS3_MAXSIMPLEUPLOADSIZEBYTES	67108864 (64 MB)

EC2 and marketplace editions

Add the property to /opt/sftpgw/application.properties on the instance:

features.file-system.aws-s3.max-simple-upload-size-bytes=67108864

Then apply the change with:

service sftpgw-admin-api restart

Connections to other clouds

The setting applies to the storage behind a cloud connection, not to the instance. An SFTP Gateway instance can have connections to more than one provider, and the S3 setting above only affects S3 connections. If this instance also writes to Azure Blob, the block size is a separate setting covered in Increasing Max Upload File Size (Azure). Google Cloud Storage needs no change; it already supports single files up to about 5 TB (see the Google Cloud article).

Important: memory

⚠️ Larger pieces use more memory. Each piece of an in-progress upload is held in the SFTP Gateway backend's memory while it transfers. As a rough guide, allow about five times the piece size of memory for each large upload happening at the same time. If you set a large piece size, or expect many large uploads at once, give the backend more memory (a larger container memory limit or a larger VM).

For the heap-sizing procedure, see Configuring VM Memory Settings and the memory section of Performance Tuning SFTP Gateway.