So, I’ve made this web scraping script for Cypress. It goes to a shop website and scrapes the listed products (product titles and prices). Namely, in the case of certain products, there is no price shown (class name: .sc-dqia0p-3), but instead, there is a message saying that the product is unavailable. In the case of the website I’m testing, the element saying that the product is not available looks like this:
<div
data-testid="product-block-unavailable-text"
class="sc-bcdv5e-9 hMrYQF"
>Ovaj proizvod više nije dostupan</div>
Basically, my goal is to set Cypress so that in cases where it cannot find the price element, instead of getting an error for a non-existing element, it should check if there is the other element above and move to the next product after that.
I’ve tried to solve this by including this else command, which you may see in the code below, but for some reason, Cypress still gets stuck at the part where it says that the price element is missing without even checking for the other element.
// Scrape the loaded products
cy.get('.sc-y4jrw3-7').each((productName, index, list) => {
// Check if the product is unavailable
cy.get('.sc-dqia0p-3').eq(index).then((priceElement) => {
// Check if the price element is available
if (Cypress.$(priceElement).length > 0) {
const title = productName.text().trim();
let price = priceElement.text().trim();
// Append "din" to the price
price = `${price} din`;
// Store product data in the products array
products.push({ title, price });
} else {
// Check if the product is unavailable
cy.get('.sc-bcdv5e-9').eq(index).then((unavailableElement) => {
const unavailableText = unavailableElement.text().trim();
if (unavailableText === 'Ovaj proizvod više nije dostupan') {
cy.log(`Product ${index + 1} is unavailable. Skipping...`);
} else {
cy.log(`Product element at index ${index} not found or unavailable. Skipping...`);
}
});
}
2
If some products don’t have a price, getting price like this cy.get('.sc-dqia0p-3').eq(index) will be out of sync with the product list if an earlier product was missing a price element.
An example in pseudo-code:
<product><price></price></product> // product index=0, price index=0
<product></product> // product index=1, price index does not exist
<product><price></price></product> // product index=2, price index=1
You want to drop cy.get('.sc-dqia0p-3').eq(index) and use product.find(price) which is the jQuery .find() method (note, it’s different to the Cypress .find() method).
const productSelector = '.sc-y4jrw3-7'
const priceSelector = '.sc-dqia0p-3'
cy.get(productSelector).each($product => {
// jQuery find() won't fail the test
const $price = $product.find(priceSelector)
if($price.length > 0) {
} else {
}
})
3