Scrapy crawl a website and lose some items

Here is my main spider class below

Scrapy crawl a website and lose some items

Here is my main spider class below

Run spider programatically integrated with a crawl lib

I have the following code that runs a spider programatically: