How I Write Web Scrapers w/ Python
The video argues that simple, one-off scraping scripts are fragile and hard to maintain, and advocates building a modular, reusable scraping system using ETL style components (extract, transform, load) to improve robustness, testability, and reusability across projects.
The library that will save your scrapers.
The video demonstrates how to implement retry logic in web scraping, using both a handcrafted decorator and a class-based approach, and then explores async retries with stamina for robust HTTP requests.
How I use Scrapy And Proxies to scrape Geo restricted sites
The video walks through a lightweight, script-based approach to scraping car data from a US site, showing how to locate JSON data in the page source, set up a Scrapy crawler as a one-off script, and export the results efficiently.
“Requests + BS4” but for 2025
The video argues that traditional web scraping with requests and Beautiful Soup is outdated for modern sites, and demonstrates a modern, asynchronous approach using proxies, TLS impersonation, and JSON LD data extraction to improve success and reduce blocking.
How I found the easiest way to scrape this site
The video walks through a practical scraping workflow on an e-commerce site, showing how to inspect network activity, locate data sources, and test API endpoints, including potential exposure of an API key and how to scale data retrieval.
Scraping these sites are easy (once you know how)
The video demonstrates how to scrape data from a web page by inspecting network requests, identifying the API endpoint, and manipulating requests and tokens (CSRF, cookies, and referer) to access vehicle data across pages, including using code to automate the process.
Scraping this NUXT site - API and Bearer found?
The video walks through discovering and exploiting a web API for vehicle data by inspecting network requests, locating a usable API endpoint, and extracting a JWT bearer token to automate repeated POST requests for data retrieval.
The modern way to scrape cloudflare sites easily
The video discusses using browser-based scraping to handle anti-bot protections, introduces a new tool (Pi Doll) that automates browser contexts and cookie handling, and demonstrates how this approach can streamline requests while mitigating detection risks.
Get daily AI recaps from
John Watson Rooney in your inbox
Get AI-powered summaries delivered to your inbox. Save hours every week while staying fully informed.