Relative Content

Tag Archive for pythonweb-scrapingpycharmweb-crawler

Icrawler unreliably downloading images

I am using icrawler on python to scrape images online. I have a list of strings download_waitlist = ["cat","dog","car","motorbike","snoop dogg"] That I want to download from the internet using icrawler. There are two issues with the package: sometimes it ends the process without having downloaded anything, meaning that there are no images in my download folder. It also simply doesn’t download the right images. I often have way too many cat images that end up getting labeled as “dog” or “car”. Any workarounds, anything I am doing wrong? Here is my downloader code: