I have a task where I need to validate up to 1000 bookmark URLs using Playwright. I need to check the validity of these URLs in parallel (ideally 3-5 concurrent checks). I’m using an external Firefox browser instance and connecting to it with Playwright.

I want to ensure the following:

1. Only create the browser instance once.
2. Check the URLs in parallel (3-8 concurrent checks).
3. Efficiently handle up to 1000 URLs.

Could someone provide guidance or a code snippet on how to achieve this?
JavaScript and Python is ok.

Additional Information

• External Browser Instance: browserless

I attempted to create multiple browser contexts, each handling a URL validation task using Playwright. My expectation was that each browser context would independently navigate to the specified URL and check its availability (HTTP status 200).

my prototype

async def check_bookmark(context, url):
    page = await context.new_page()
    try:
        await page.goto(url)
        status = await page.evaluate('() => document.readyState')
        if status == 'complete':
            print(f"{url} is valid")
        else:
            print(f"{url} is invalid")
    except Exception as e:
        print(f"{url} is invalid: {str(e)}")
    finally:
        await page.close()

async def main():
    bookmark_urls = ["https://example.com"]

    async with async_playwright() as p:
        browser = await p.firefox.connect('ws://localhost:3000/playwright/firefox')
        
        contexts = []

        for i in range(0, 3):
            context = await browser.new_context()
            contexts.append(context)

        tasks = []
        num_contexts = len(contexts)
        num_urls = len(bookmark_urls)
        urls_per_context = num_urls // num_contexts

        for i in range(num_contexts):
            start_index = i * urls_per_context
            end_index = start_index + urls_per_context
            urls_subset = bookmark_urls[start_index:end_index]

            for url in urls_subset:
                tasks.append(check_bookmark(contexts[i], url))

        await asyncio.gather(*tasks)

        await browser.close()

if __name__ == '__main__':
    asyncio.run(main())

But this always gave me this error: Target page, context or browser has been closed