Understanding Blob Storage Events from SFTP Gateway File Uploads

Overview

When uploading files through SFTP Gateway to Azure Blob Storage, you may notice multiple storage events triggered for a single file upload. This is expected behavior due to how the SFTP protocol translates file operations to Azure Blob Storage APIs.

This article explains why these events occur and provides solutions for filtering them in your Azure Functions or automation workflows.

How Many Events Are Generated?

Each file upload through SFTP Gateway generates 2-3 Blob Storage events:

Initial blob creation with metadata - A 0-byte blob is created with timestamp metadata (mtime, ctime, atime, md5).
Blob data upload - The blob is overwritten with the actual file content.
Metadata update - Blob metadata is updated after the upload completes (timestamps are updated if preservation is enabled).

Why Does This Happen?

This behavior is inherent to how SFTP operations are translated to Azure Blob Storage APIs:

CREATE FILE command - When an SFTP client initiates an upload, it sends a "create file" command, which SFTP Gateway translates into creating a 0-byte blob in Azure with initial metadata.
PUT FILE command - Once the entire file data is transferred, the blob is overwritten with the actual content.
SET ATTRIBUTES command - After the upload completes, SFTP Gateway updates metadata and optionally updates timestamps if preservation is enabled.

Note: Different SFTP clients may behave slightly differently, which can affect the exact timing of events. This is normal and depends on the client's implementation of the SFTP protocol.

Azure Blob Storage Event Details

For a single file upload to Azure Blob Storage, you'll see:

Event Type	Description
`Microsoft.Storage.BlobCreated`	Initial 0-byte blob creation with metadata
`Microsoft.Storage.BlobCreated`	Blob content uploaded (overwrites the 0-byte blob)
`Microsoft.Storage.BlobPropertiesUpdated`	Metadata update (upload complete)

Example Event Grid events:

1. Microsoft.Storage.BlobCreated                uploads/Thorn_Tech.pdf
2. Microsoft.Storage.BlobCreated                uploads/Thorn_Tech.pdf
3. Microsoft.Storage.BlobPropertiesUpdated      uploads/Thorn_Tech.pdf

How to Handle Multiple Events in Azure Functions

If these duplicate events are causing issues in your Azure Functions, here are the recommended approaches:

Solution 1: Filter by Blob Size (Recommended)

The simplest and most reliable solution is to ignore events for 0-byte or very small blobs (< 1 KB), as the initial creation event will always be minimal in size.

Python Example:

import logging
import azure.functions as func

def main(blob: func.InputStream):
    logging.info(f"Python blob trigger function processed blob")
    logging.info(f"Name: {blob.name}")
    logging.info(f"Blob Size: {blob.length} bytes")
    
    # Ignore small blobs
    if blob.length < 1024:  # Less than 1 KB
        logging.info(f"Ignoring small blob event: {blob.name} ({blob.length} bytes)")
        return
    
    # Process the actual blob upload
    logging.info(f"Processing blob: {blob.name}")
    process_blob(blob)

def process_blob(blob):
    """Your file processing logic here."""
    logging.info(f"Processing blob: {blob.name}")

Why this works:

The initial blob creation is always 0 bytes
Actual file uploads will be larger than 1 KB in most cases
Simple, fast, and doesn't require additional API calls
Works reliably across all SFTP clients

Solution 2: Add a Delay Before Processing

For simple workflows with files under 25 MB, adding a brief delay before processing can ensure you're working with the final version of the blob.

Python Example:

import logging
import time
import azure.functions as func

def main(blob: func.InputStream):
    logging.info(f"Python blob trigger function processed blob")
    logging.info(f"Name: {blob.name}")
    
    # Wait 15 seconds to ensure all events have completed
    time.sleep(15)
    
    # Process the blob
    logging.info(f"Processing blob: {blob.name}")
    process_blob(blob)

def process_blob(blob):
    """Your file processing logic here."""
    logging.info(f"Processing blob: {blob.name}")

Note: This approach works well for smaller files (under 25 MB) where the upload completes quickly. For larger files or high-throughput systems, use Solution 1 instead.

Recommended Approach

For most use cases: Use Solution 1 (filter by blob size) - it's simple, reliable, efficient, and doesn't require additional Azure Storage API calls or delays.

For simple, low-traffic workflows: Use Solution 2 (delay approach) - only suitable for small files and scenarios where a 15-second delay is acceptable.

Viewing Events in Azure Monitor

To observe these events for debugging:

Option 1: Using Storage Analytics Logs (Legacy)

Go to your Storage Account → Monitoring → Diagnostic settings
Click Add diagnostic setting
Select StorageWrite logs
Choose destination (Log Analytics workspace recommended)
Save the configuration

Option 2: Using Event Grid (Recommended)

Event Grid provides real-time event delivery and is the modern approach for monitoring blob events.

Go to your Storage Account → Events
Click + Event Subscription
Configure:
- Name: Give your subscription a name
- Event Schema: Event Grid Schema
- Filter to Event Types: Select Blob Created and Blob Properties Updated
- Endpoint Type: Choose webhook, Azure Function, or Service Bus
Create the subscription

Query Events in Log Analytics

If you configured diagnostic settings to send logs to Log Analytics:

StorageBlobLogs
| where AccountName == "yourstorageaccount"
| where OperationName in ("PutBlob", "PutBlockList", "SetBlobMetadata", "SetBlobProperties")
| where Uri contains "uploads/"
| project TimeGenerated, OperationName, Uri, StatusText, ContentLength
| order by TimeGenerated asc

To filter for a specific file:

StorageBlobLogs
| where AccountName == "yourstorageaccount"
| where Uri contains "your-file.pdf"
| project TimeGenerated, OperationName, Uri, ContentLength
| order by TimeGenerated asc

Azure Function Configuration Tips

Function Timeout Considerations

Set appropriate timeouts based on your solution.

Recommended timeouts:

Solution 1 (file size filter): 60 seconds
Solution 2 (delay approach): Minimum 60 seconds (15s delay + processing time)

Configure in host.json:

{
  "version": "2.0",
  "functionTimeout": "00:02:00",
  "extensions": {
    "blobs": {
      "maxDegreeOfParallelism": 4
    }
  }
}

Permissions Required

Your Function App's managed identity or connection string needs these permissions:

Storage Blob Data Reader - To read blob content and metadata
Storage Blob Data Contributor - If you need to modify blobs

Assign via Azure Portal: Storage Account → Access Control (IAM) → Add role assignment

Or using Azure CLI:

az role assignment create \
  --role "Storage Blob Data Contributor" \
  --assignee <function-app-principal-id> \
  --scope /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>

Additional Notes

This behavior is expected and cannot be disabled, as it's how the SFTP protocol translates to Azure Blob Storage APIs.
The extra events do not increase storage costs (only one blob exists in Azure).
Timestamp preservation can be disabled in your SFTP client settings if you don't need original timestamps, but the three-event pattern will still occur.
Different SFTP clients may produce slightly different event patterns, but the core behavior remains the same.
SFTP Gateway sets custom metadata on blobs including mtime, ctime, atime, and md5 for file verification.

Need Help?

If you're still experiencing issues with duplicate events or need assistance implementing these filtering strategies, please contact our support team at support@thorntech.com.

SFTP Gateway Support

3.0