Publications by Python | datawookie

Scrapy with a Rotating Tor Proxy

This post shows an approach to using a rotating Tor proxy with Scrapy.I’m using the scrapy-rotating-proxies download middleware package to rotate through a set of proxies, ensuring that my requests are originating from a selection of IP addresses. However, I need to have those IP addresses evolve over time too, so I’m using the Tor network.Se...

2613 sym