I am trying to access data(quote value) from an E-commerce website using the ‘requests’ library in python. The problem I have is that the cookies in the website are dynamic. And my code requires a header to get a response.
I can open the website and scrape it but to do that I need to copy the header details from the response header. However I need to automate this process so that I don’t need to manually put the cookie in every time I want to scrape. This the link “https://www.nseindia.com/get-quotes/equity?symbol=RELIANCE”. I am trying to get the ‘Intraday chart’ data so I can store it in a DataFrame and plot it.

I am a beginner and I have never web scraped before.

This is what I have tried so far.

import requests
import pandas as pd

# I take this data from the website response headers

headers = {
    'Accept': 'application/json, text/javascript, */*; q=0.01',
    'Accept-Encoding': 'gzip, deflate, br, zstd',
    'Accept-Language': 'en-US,en;q=0.5',
    'Cookie' : 'Cookie Value'
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36',
    'X-Requested-With': 'XMLHttpRequest'
}

response = requests.get(url = 'https://www.nseindia.com/api/chart-databyindex?index=RELIANCEEQN', headers = headers)

response.json()['grapthData']

reliance = pd.DataFrame(response.json()['grapthData'])

reliance.columns = ['Timestamp', 'Price']

reliance['Timestamp'] = pd.to_datetime(reliance['Timestamp'], unit = 'ms' )

reliance.plot(x = 'Timestamp', y = 'Price')