I have the following situation. I send a request from 3 sources (postman, cURL and python requests library). For Postman and cURL I receive the same response but for python requests library I get a different response and I don’t understand the difference:
In Postman I send the request without cookies and without follow redirects. All the requests use the same IP address from the same machine.
Versions:
cURL
curl 8.6.0 (x86_64-apple-darwin23.0) libcurl/8.6.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.61.0
Python
Python 3.7
requests 2.25.1
OS
macOS 14.5 (x64)
cURL command
curl --location '<URL>'
--header 'host: <HOST>'
--header 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36'
--header 'Accept-Encoding: gzip, deflate'
--compressed > test.html
Python requests code
import requests
url = "<URL>"
payload = {}
headers = {
'host': '<HOST>',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36',
'Accept-Encoding': 'gzip, deflate'
}
response = requests.request("GET", url, headers=headers, data=payload)
print(response.text)
I tried to check which http protocol is use by all these methods and I found that all use HTTP 1.1
I don’t understand what is the difference between these methods(cURL and requests) so I receive different response.