When visiting http://www.google.com in a browser located in the US, it should be redirected to https://www.google.com. However, this does not appear to work when using Python Requests, as demonstrated in the code below.
import requests
r = requests.get("https://www.google.com", allow_redirects=True)
print(r.url)
print(r.history)
r = requests.get("http://www.google.com", allow_redirects=True)
print(r.url)
print(r.history)
r = requests.get("https://google.com", allow_redirects=True)
print(r.url)
print(r.history)
r = requests.get("http://google.com", allow_redirects=True)
print(r.url)
print(r.history)
r = requests.get("http://github.com", allow_redirects=True)
print(r.url)
print(r.history)
The output is:
https://www.google.com/
[]
http://www.google.com/
[]
https://www.google.com/
[<Response [301]>]
http://www.google.com/
[<Response [301]>]
https://github.com/
[<Response [301]>]
You can see that the second request was not redirected from HTTP to HTTPS using Python Requests, however, the redirection does occur in my Chromium browser. In contrast, the request made to http://github.com/ was responded with a redirection response, which was followed. Using Chromium’s Developer Tools, I observed that the very first request made to http://www.google.com is already converted to https://www.google.com, which might be something only done by the browser but not Python Requests. An answer regarding HTTPS redirection on Super User suggests that this might be related to HSTS?
Therefore, I would like to know why Python Requests does not do HTTP to HTTPS redirection for certain websites, and how can I make such redirection work as if I am using a browser?
I am using Python 3.10.12, requests==2.32.3, and I am comparing it to Chromium 126.0.6478.126.