I am trying to write a script that goes to Hacker News and gets the first 100 news articles but I am getting .itemlist not found.
My code:
(async () => {
try {
// Launch a browser
const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('https://news.ycombinator.com/newest', { waitUntil: 'networkidle' });
await page.waitForTimeout(5000);
// Print the page content for debugging
const content = await page.content();
console.log('Page content:', content);
// Check if the .itemlist element is present
const isItemListPresent = await page.$('.itemlist') !== null;
console.log('.itemlist element present:', isItemListPresent);
if (!isItemListPresent) {
throw new Error('.itemlist element not found');
}
// Wait for the .itemlist element to be visible
await page.waitForSelector('.itemlist', { timeout: 60000 });
// Extract the timestamps of the first 100 articles
const articleTimestamps = await page.$$eval('.itemlist .athing .age', elements =>
elements.slice(0, 100).map(el => el.innerText)
);
})();
Output: .itemlist element present: false
I think I am supposed to use a different selector instead of .itemlist but not sure which one