I’m using gdal_translate to download a specific size overview of the COG file from HLS dataset. Sometimes download ends with error like http 403.

I’m trying to force gdal_translate to do 50 attempts to download a given URL that returns 403 error. With use of GDAL_HTTP_MAX_RETRY and GDAL_HTTP_RETRY_CODES. But it does only one attmpt to get URL.

A simplified request url:

gdal_translate --debug on --config GDAL_HTTP_COOKIEFILE /tmp/cookie --config GDAL_HTTP_COOKIEFILE /tmp/cookie  --config GDAL_HTTP_COOKIEJAR /tmp/cookie --config GDAL_DISABLE_READDIR_ON_OPEN ON --config CPL_VSIL_CURL_ALLOWED_EXTENSIONS TIF --config GDAL_HTTP_MAX_RETRY 50 --config GDAL_HTTP_RETRY_CODES ALL --config GDAL_HTTP_UNSAFESSL YES "https://d1nklfio7vscoe.cloudfront.net/head-2d2df3a34830d5223d1e9547cd713408/lp-prod-protected.s3.us-west-2.amazonaws.com/HLSS30.020/HLS.S30.T39UWA.2018020T080231.v2.0/HLS.S30.T39UWA.2018020T080231.v2.0.B04.tif"  /tmp/output.tif

The output is:

HTTP: Fetch(https://d1nklfio7vscoe.cloudfront.net/head-2d2df3a34830d5223d1e9547cd713408/lp-prod-protected.s3.us-west-2.amazonaws.com/HLSS30.020/HLS.S30.T39UWA.2018020T080231.v2.0/HLS.S30.T39UWA.2018020T080231.v2.0.B04.tif)
HTTP: libcurl/8.5.0 OpenSSL/3.0.13 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 (+libidn2/2.3.7) libssh/0.10.6/openssl/zlib nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
ERROR 1: HTTP error code : 403

So, only one attempt to get file is made althogh GDAL_HTTP_RETRY_CODES ALL is passed.

Any ideas how to make GDAL_HTTP_MAX_RETRY and GDAL_HTTP_RETRY_CODES work as expected?

My gdalinfo --version : GDAL 3.8.4, released 2024/02/08
P.S It uses libcurl under the hood and it must be curl, as I need /vsicurl/https:// schema in a not simplified version of my request.