I’m trying to build a multi-search tool that searches a list of websites and outputs results from all of them on one page as an array.

The NPM package node-fetch works fine for most of the sites: fetch the desired URL, send the HTML to the front end and then use a DOM Parser to search through the HTML for the elements I want to extract data from.

Now, the last site I’m trying to search has un-rendered content which appears in the fetched HTML as {{Title}} for example instead of the actual text title which I want to extract.

I realise the content is unrendered so after some searching I found Puppeteer and I’m trying to use it as below:

const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.goto(url)
const HTML = await page.content()
await browser.close()

However, the output HTML variable still contains the un-rendered content.

I saw another StackOverflow post that recommended changing line 3 to this: await page.goto(url, { waitUntil: 'networkidle' }) however this just throws Error: Unknown value for options.waitUntil: networkidle.

Any help appreciated.