In this video, we look at how to bypass any blockers, anti-bots, and anything else that might be stopping us from scraping a website using Python Scrapy by using a mixture of User Agents and Proxies.
The list of different proxy providers: https://scrapeops.io/proxy-providers/comparison/
The code we use is available for download here: https://github.com/ScrapeOps/python-scrapy-playbook/tree/master/1_Beginner_Series
00:00 - Intro
00:34 - Why we get blocked?
01:46 - What are User Agents?
03:57 - Adding multiple Scrapy User Agents to a Scrapy project
07:06 - Rotating your IP address using free Proxies
13:21 - Concurrent requests when scraping with Proxies
14:36 - Outro
