Blob Storage Upload Limitations · SFTP Gateway Support

Overview

There currently is an issue when uploading files around 200 GB.

The error originates from the default block size in the Azure SDK. There is a 50,000 block limit, and the default block size is 4 MB. This results in a file size limit of 195 GB.

We are working to address this issue in version 3.3.0 of SFTP Gateway by exposing a block size configuration option.

The Error Message

When uploading a 200 GB or larger file you may see this response in /opt/sftpgw/log/application.log:

2022-03-24 10:48:00 - /home/robtest/home/robtest/uploads/large-file upload result is:
ERROR: Retry policy did not allow for a retry: 
HTTP status code=409, 
Exception=The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.
ErrorCode: BlockCountExceedsLimit
The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.