I’m scrapping an e-commerce website with thousands of objects. BeautifulSoup with request does the stuff quick. But only to reveal the phone numbers (<a> tag with href references to js script) I have to use selenium running a web driver which becomes a bottleneck.
What effective method to make this kind of parsing relatively quick ? Mb some common approaches like multiprocessing?