Web Scraping
Tools & Libraries Requests, Beautiful Soup, Crawlee, httpx and other scraping tools, and how they compare. Browsers & Rendering Playwright, Puppeteer, Selenium, headless browsers, CDP and scraping JavaScript-heavy sites. Crawling at Scale Architecture, queues, scheduling, monitoring, cost and reliability for large crawls. Data Extraction & Parsing Selectors, XPath, schemas, automatic and AI-powered extraction, and turning pages into structured data. Guides & Fundamentals Tutorials, how-tos and the basics of web scraping, from first request to production. Data Pipelines & AI Datasets Cleaning, storing and delivering scraped data, including datasets for RAG and LLMs. Anti-Bot & Unblocking Bans, CAPTCHAs, TLS and browser fingerprinting, proxies, and how sites detect scrapers.
| Topic | Replies | Views | Activity |
|---|