I’m working in a workflow that needs to download zip files from a SFTP server. Those files have, in average, about 16MB.

To do that, I’m using airflow SFTPHook. The problem is that when I try to use .retrieve_file() function, it keeps running forever and finally fails to download the file with the following error:

File "/home/airflow/.local/lib/python3.7/site-packages/paramiko/sftp_client.py", line 854, in _read_response
    raise SSHException("Server connection dropped: {}".format(e))
paramiko.ssh_exception.SSHException: Server connection dropped:

Here’s the piece of code I’m using to try to download the file:

from airflow import DAG
from airflow.providers.sftp.hooks.sftp import SFTPHook

import datetime

# Download file from SFTP
hook = SFTPHook(ssh_conn_id='my_connection')
remote_path = '/mypath/file.zip'
local_path = '/tmp/file.zip'
hook.retrieve_file(remote_full_path=remote_path, local_full_path=local_path)
print("execution OK")

Does SFTPHook have a file size limitation? Is 16MB too to handle? Do anyone knows any other option I could use for this purpose?