aws cp not working in cron

I just had a problem downloading some files from Amazon Web Services using the aws cp command. The script ran fine as the root user but did not when run in roots cron tab. A normal copy command that took just seconds, timed out with the error:

HTTPSConnectionPool(host='s3.eu-central-1.amazonaws.com', port=443): Max retries exceeded with url: /bla.txt (Caused by ConnectTimeoutError(, 'Connection to s3.eu-central-1.amazonaws.com timed out. (connect timeout=60)'))
Completed 1 part(s) with ... file(s) remaining

I came across Kris Jordan‘s excellent tips and finally got the idea that it may be my proxy environment variables that were not getting accessed.

I fixed the issue by adding the environment variable directly in the script I was running.

export https_proxy="http://username:password@proxy.example.com:12345"
export http_proxy="http://username:password@proxy.example.com:12345"

Note: that is case sensitive, so use https_proxy/http_proxy and not HTTPS_PROXY/HTTP_PROXY.

This entry was posted in General. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *