Sometimes this works and sometimes it doesn’t. I don’t understand why. Any ideas or suggestions? The tags are correct.
URL = ‘https://www.amazon.com/Bose-QuietComfort-Cancelling-Headphones-Bluetooth/dp/B0CCZ26B5V/ref=sr_1_3?crid=2491RYA872DT3&dib=eyJ2IjoiMSJ9.nt7VYtXp5pWj7xfzTrxzsfBi7xDLEcL_Ph-ke9myd8-dXCeIaAXuKVBFQXNPD2c0_SjLQkXBe_aMyr648OQPVUI6rX2eQnp823zAKljvkC4NhQ6756Bw4s2Gc8AG4bEJx9ndltFcrjCeK-tgaVlbK77agZb5oykKzvYDhkCm2ycTiIMfn5-yPGbRgzYWX4Juzb1SrfBu2lj_kz5dXKyD1-P2Hm8hNX3F_VrpCGqGRKo.cGaDMSRHm-DXr5z4IFZW5juUZ8mbIgzEYcHxnG3kPlA&dib_tag=se&keywords=headphones&qid=1718911530&sprefix=head%2Caps%2C452&sr=8-3&th=1’
headers = {“User-Agent”: “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36”}
page = requests.get(URL, headers=headers)
soup1 = BeautifulSoup(page.content,”html.parser”)
soup2 = BeautifulSoup(soup1.prettify(),”html.parser”)
title = soup2.find(id=’productTitle’).get_text()
price = soup2.find(class_=’aok-offscreen’).get_text()
price = price.strip()[1:]
title = title.strip()
print(title)
print(price)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[64], line 20
16 soup1 = BeautifulSoup(page.content,"html.parser")
18 soup2 = BeautifulSoup(soup1.prettify(),"html.parser")
---> 20 title = soup2.find(id='productTitle').get_text()
22 price_element = soup.find(class_='aok-offscreen')
23 if price_element:
AttributeError: 'NoneType' object has no attribute 'get_text'
I tried looking up the documentation on get_text but I still can’t figure it out. I have noticed that if I remove the print(price) and run it a few times it will work.
Dallin Thomson is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.