Thiết kế website giá rẻ

Question

my post request is working fine without scrapeops proxies but client said not to disable proxies.My request requires both cookies and headers along with payload data to get json response, here are my headers and cookies:

cookies = {
            'tg_session_25008_5012': '^TAmfhlbvqGGm3Vo9KBWxZJqolgCqcCoXPEO9qDwlTkJyhEFcx7zrQeseu8KOyFsEjx3D3GAbMVOcU7Agx7LOq90ub7wGGK354CKkKngCOH4=',
            'tg_session': '^TAmfhlbvqGGm3Vo9KBWxZJqolgCqcCoXPEO9qDwlTkJyhEFcx7zrQeseu8KOyFsEjx3D3GAbMVOcU7Agx7LOq90ub7wGGK354CKkKngCOH4=',
            'tg_rft': '^mHOwVU4ei7NrQmF6n2llJb/gAewiggqW8DkdZR4sm9lKjwoWo3Nf4D4dN8aqigFzzljy446oEvDpcUWUpgMSEPcZ3tSH90ojDHue1sa6zgo=',
            'tg_rft_mvc': '1RJwHo4bG8T09Qfvi1Y9tZfyUGF-1x_sH9RnYG840X_fXpW418Lv--2T9duP2H-mrx7OFxrxOIzX3CJ5Zd8NJgIQ0fV_7vn-LEaw4mFgapF1QEHM4C7gByEUjlYhIR55MY_IJA2',
            'geo-country': 'PK',
            'sat_track': 'true',
            's_lv_s': 'First%20Visit',
            'AMCVS_73FAC51D54C72AE50A4C98BC%40AdobeOrg': '1',
            's_ecid': 'MCMID%7C75522721953618165144521412867917598680',
            's_visit': '1',
            's_mtouch': 'Internal',
            's_cc': 'true',
            'aam_uuid': '75493351460292711974518490112521156743',
            'AAMSID': 'AAM%3D21740122%3A21739909',
            'AMCV_73FAC51D54C72AE50A4C98BC%40AdobeOrg': '770003774%7CMCIDTS%7C19984%7CMCMID%7C75522721953618165144521412867917598680%7CMCAAMLH-1727203260%7C3%7CMCAAMB-1727203260%7CRKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y%7CMCOPTOUT-1726605660s%7CNONE%7CMCAID%7CNONE%7CvVersion%7C5.5.0%7CMCCIDH%7C-1236019896',
            's_ppn': 'https%3A%2F%2Fwww.ubs.com%2Fglobal%2Fen%2Fcareers%2Fmeet-us.html',
            's_gpv_url2': 'https%3A%2F%2Fwww.ubs.com%2Fglobal%2Fen%2Fcareers.html',
            's_gpv_channel': 'global%3Acareers%3Ameet%20us%3Aresponsive_content',
            's_gpv_url': 'https%3A%2F%2Fwww.ubs.com%2Fglobal%2Fen%2Fcareers%2Fmeet-us.html',
            's_nr': '1726598461275-New',
            's_lv': '1726598461275',
            's_ppvl': 'https%253A%2F%2Fwww.ubs.com%2Fglobal%2Fen%2Fcareers%2Fmeet-us.html%2C53%2C5%2C1741%2C1366%2C641%2C1366%2C768%2C1%2CP',
            'scrollTracking': 'true',
            's_ht': '1726598485291',
            's_hc': '1%7C0%7C1%7C0%7C2',
            's_ppv': 'https%253A%2F%2Fwww.ubs.com%2Fglobal%2Fen%2Fcareers%2Fmeet-us.html%2C100%2C53%2C3295%2C1366%2C641%2C1366%2C768%2C1%2CP',
            's_sq': 'ubs-loglive%3D%2526c.%2526a.%2526activitymap.%2526page%253Dhttps%25253A%25252F%25252Fwww.ubs.com%25252Fglobal%25252Fen%25252Fcareers%25252Fmeet-us.html%2526link%253DOK%2526region%253Ddoc%2526.activitymap%2526.a%2526.c%2526pid%253Dhttps%25253A%25252F%25252Fwww.ubs.com%25252Fglobal%25252Fen%25252Fcareers%25252Fmeet-us.html%2526oid%253DOK%2526oidt%253D3%2526ot%253DSUBMIT',
            'ubs_cookie_settings_2.0.4': '0-4-3-2-1',
            'tg_rft': '^mHOwVU4ei7NrQmF6n2llJb/gAewiggqW8DkdZR4sm9lKjwoWo3Nf4D4dN8aqigFzzljy446oEvDpcUWUpgMSEPcZ3tSH90ojDHue1sa6zgo=',
        }

        headers = {
            'Accept': 'application/json, text/plain, */*',
            'Accept-Language': 'en-US,en;q=0.9',
            'Connection': 'keep-alive',
            'Content-Type': 'application/json;charset=UTF-8',
            # 'Cookie': 'tg_session_25008_5012=^TAmfhlbvqGGm3Vo9KBWxZJqolgCqcCoXPEO9qDwlTkJyhEFcx7zrQeseu8KOyFsEjx3D3GAbMVOcU7Agx7LOq90ub7wGGK354CKkKngCOH4=; tg_session=^TAmfhlbvqGGm3Vo9KBWxZJqolgCqcCoXPEO9qDwlTkJyhEFcx7zrQeseu8KOyFsEjx3D3GAbMVOcU7Agx7LOq90ub7wGGK354CKkKngCOH4=; tg_rft=^mHOwVU4ei7NrQmF6n2llJb/gAewiggqW8DkdZR4sm9lKjwoWo3Nf4D4dN8aqigFzzljy446oEvDpcUWUpgMSEPcZ3tSH90ojDHue1sa6zgo=; tg_rft_mvc=1RJwHo4bG8T09Qfvi1Y9tZfyUGF-1x_sH9RnYG840X_fXpW418Lv--2T9duP2H-mrx7OFxrxOIzX3CJ5Zd8NJgIQ0fV_7vn-LEaw4mFgapF1QEHM4C7gByEUjlYhIR55MY_IJA2; geo-country=PK; sat_track=true; s_lv_s=First%20Visit; AMCVS_73FAC51D54C72AE50A4C98BC%40AdobeOrg=1; s_ecid=MCMID%7C75522721953618165144521412867917598680; s_visit=1; s_mtouch=Internal; s_cc=true; aam_uuid=75493351460292711974518490112521156743; AAMSID=AAM%3D21740122%3A21739909; AMCV_73FAC51D54C72AE50A4C98BC%40AdobeOrg=770003774%7CMCIDTS%7C19984%7CMCMID%7C75522721953618165144521412867917598680%7CMCAAMLH-1727203260%7C3%7CMCAAMB-1727203260%7CRKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y%7CMCOPTOUT-1726605660s%7CNONE%7CMCAID%7CNONE%7CvVersion%7C5.5.0%7CMCCIDH%7C-1236019896; s_ppn=https%3A%2F%2Fwww.ubs.com%2Fglobal%2Fen%2Fcareers%2Fmeet-us.html; s_gpv_url2=https%3A%2F%2Fwww.ubs.com%2Fglobal%2Fen%2Fcareers.html; s_gpv_channel=global%3Acareers%3Ameet%20us%3Aresponsive_content; s_gpv_url=https%3A%2F%2Fwww.ubs.com%2Fglobal%2Fen%2Fcareers%2Fmeet-us.html; s_nr=1726598461275-New; s_lv=1726598461275; s_ppvl=https%253A%2F%2Fwww.ubs.com%2Fglobal%2Fen%2Fcareers%2Fmeet-us.html%2C53%2C5%2C1741%2C1366%2C641%2C1366%2C768%2C1%2CP; scrollTracking=true; s_ht=1726598485291; s_hc=1%7C0%7C1%7C0%7C2; s_ppv=https%253A%2F%2Fwww.ubs.com%2Fglobal%2Fen%2Fcareers%2Fmeet-us.html%2C100%2C53%2C3295%2C1366%2C641%2C1366%2C768%2C1%2CP; s_sq=ubs-loglive%3D%2526c.%2526a.%2526activitymap.%2526page%253Dhttps%25253A%25252F%25252Fwww.ubs.com%25252Fglobal%25252Fen%25252Fcareers%25252Fmeet-us.html%2526link%253DOK%2526region%253Ddoc%2526.activitymap%2526.a%2526.c%2526pid%253Dhttps%25253A%25252F%25252Fwww.ubs.com%25252Fglobal%25252Fen%25252Fcareers%25252Fmeet-us.html%2526oid%253DOK%2526oidt%253D3%2526ot%253DSUBMIT; ubs_cookie_settings_2.0.4=0-4-3-2-1; tg_rft=^mHOwVU4ei7NrQmF6n2llJb/gAewiggqW8DkdZR4sm9lKjwoWo3Nf4D4dN8aqigFzzljy446oEvDpcUWUpgMSEPcZ3tSH90ojDHue1sa6zgo=',
            'Origin': 'https://jobs.ubs.com',
            'RFT': 'XggmEZZl_onTMvzo1OXHKL7GGQ494onDiAt8VQis0jk4M6IhNbtF_PM1prnkjQv_0ikrKd7kp9t4lsz1TzU5Dd1IIhsKZgw0UBz1pkh50ysHchN16nFJlH5VI1wAbowucA_xcQ2',
            'Referer': 'https://jobs.ubs.com/TGnewUI/Search/home/HomeWithPreLoad?partnerid=25008&siteid=5012&PageType=searchResults&SearchType=linkquery&LinkID=6015',
            'Sec-Fetch-Dest': 'empty',
            'Sec-Fetch-Mode': 'cors',
            'Sec-Fetch-Site': 'same-origin',
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36',
            'sec-ch-ua': '"Google Chrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"',
            'sec-ch-ua-mobile': '?0',
            'sec-ch-ua-platform': '"Windows"',
        }

here is my request:

yield scrapy.Request(url, method='POST', callback=self.parse_detail, body=json.dumps(payload), headers=headers, cookies=cookies, meta={'sops_keep_headers': True, })

here is the response when i use proxies:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid Header</h2>
<hr><p>HTTP Error 400. The request has an invalid header name.</p>
</BODY></HTML>

simply when i disable the proxies and send request it works proper and json response i received but when i use scrapeops proxies it it give Bad Request

Thiết kế website giá rẻ

Danh mục

I am having a Post request “HTTP Error 400. The request has an invalid header name.” in scrapy while using Scrapeops proxies