I am trying to learn asynchronous programming in Python.
I have a simple get request that returns status code 200 when using the Requests modules.
When trying to reproduce the same request using AIOHTTP, I get a 403 status code response.
My question is : How does AIOHTTP differentiate itself from Requests when it comes to constructing an http request?
I believe that since I am passing the same parameters to both requests, there might be differences in the way the requests are sent, resulting in the AIOHTTP request failing.. Possibly something to do with the headers or cookies..
import aiohttp
import asyncio
import requests
url = "https://spectate-web.888sport.com/spectate/sportsbook-req/getTournamentMatches/ice-hockey/united-states-of-america/nhl"
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.79 Safari/537.36',
'Accept-Encoding': 'gzip, deflate',
'Accept': '*/*',
'Connection': 'keep-alive',
'Cookie': 'odds_format=DECIMAL; optimizelyEndUserId=oeu1712128176635r0.5640445223678774; _cs_c=0; FPID=FPID2.2.p9QFQEw3wxQgGubYSYQTLEyXHhaPlG6hnxGm5JhFQZs%3D.1712128177; OptanonAlertBoxClosed=2024-04-03T07:09:47.833Z; _gcl_au=1.1.303053898.1712128188; OptanonConsent=isGpcEnabled=0&datestamp=Wed+Apr+03+2024+00%3A09%3A47+GMT-0700+(Pacific+Daylight+Time)&version=202209.2.0&isIABGlobal=false&hosts=&consentId=9f33e415-6ad2-41d4-9ce8-5aa5d4b9c619&interactionCount=1&landingPath=NotLandingPage&groups=C0001%3A1%2CC0002%3A1%2CC0003%3A1%2CC0004%3A1; FPAU=1.2.1880634315.1712128189; __qca=P0-1150796283-1712128188063; blueID=b6302533-2e9a-4ce6-8220-9c82c65df18a; _fbp=fb.1.1712128192008.2093727263; _scid=fd08dc38-6af8-4912-d93b-898a1a94bc17; login_deviceuuid=37d22c5f-2f3e-4dff-a627-879c9dc45870; anon_hash=05c6744a8af2029e8018973278168195; spectate_session=780bc31d-1423-4820-955e-2c1bff1e598a%3Aanon; 888Attribution=1; 888Cookie=lang%3Den%26OSR%3D1927680; 888TestData=%7B%22strategysource%22%3A%22previousvisit%22%2C%22currentvisittype%22%3A%22Unknown%22%2C%22referrer%22%3A%22https%3A%2F%2Fwww.google.com%2F%22%2C%22orig-lp%22%3A%22https%3A%2F%2Fwww.888sport.com%2Fbaseball%2Funited-states-of-america%2Fmajor-league-baseball-t-324139%2F%22%2C%22publisher%22%3A%22SearchEngine%22%2C%22strategy%22%3A%22UnknownStrategy%22%2C%22datecreated%22%3A%222024-05-09T18%3A46%3A37.924Z%22%2C%22expiredat%22%3A%22Thu%2C%2016%20May%202024%2018%3A46%3A00%20GMT%22%2C%22datemodified%22%3A%222024-05-13T23%3A59%3A03.232Z%22%2C%22modifiedcounter%22%3A%224%22%7D; spectate_client_ver=2.71; bbsess=zi4-oo0BBzStlX6jecq76Ohha-x; lang=enu; _cs_cvars=%7B%7D; _gid=GA1.2.388250393.1715644745; _gat_UA-125725186-7=1; FPGSID=1.1715644745.1715644745.G-15CNXYXJ7R.lERb7naix8ZqN0n-RV2TSA; FPLC=os%2FV%2FxiV%2BsUjh9NAS1aCO6Z7OGcyMwfxMEOAXXjQ8fFn2TwmxnMIS3TNIde0PK%2B5Exqd5ou1ztzKH5uqLKIXj9Yt%2Bk7DCeFKK7ni9z%2Bx1LCHs%2FzC6WTKZkn4EoIUjQ%3D%3D; _cs_id=8ce8877b-5380-a03e-ec85-8434a4d28294.1712128178.29.1715644748.1715644744.1708336297.1746292178098.1; _cs_s=2.5.0.1715646548363; _uetsid=c6fa0430118411ef9efc39c590568d56; _uetvid=5ac21390f2fd11ee87406f760e46580c; _ga=GA1.2.957392554.1712128177; _ga_15CNXYXJ7R=GS1.1.1715644744.30.1.1715644756.0.0.1631588786'
}
resp = requests.get(url=url, headers=headers)
print(resp.status_code)
#Output 200
async def main():
async with aiohttp.ClientSession() as session:
async with session.get(url, headers=headers) as response:
print(response.status)
asyncio.run(main())
#Output 403
I have tried printing the headers of both requests to make sure they are identical using
print(resp.request.headers)
for Requests
and print(response._request_info)
for AIOHTTP
I noticed the header “Host” was automatically added by Requests but not by AIOHTTP.
After adding it manually in the headers, I am still getting error 403.
What else should I look into?
Any help is greatly appreciated.
Kevin O is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.