Writing Scrapers

• Getting Started Writing Scrapers
  • Python
  • Understanding GitHub
  • Scraping Basics
  • Getting Started
• Creating a New Scraper
• Writing a Person Scraper
  • Special notes about People scrapers
  • Target Data
  • Creating a New Person scraper
  • Committees and Memberships
  • Scraper Example
  • Special notes regarding Posts, Memberships and Districts
• Writing an Events Scraper
  • Target Data
  • Creating a new Events scraper
• Writing a Bill Scraper
  • Target Data
  • Overview
  • Scraping Votes
• Running the Scraper
  • Validating Data
• Submitting a Pull Request
  • Fork the repo you want to contribute to
  • Submit a pull request
  • Best Practice
  • Keeping your branch up to date
  • Checking what you’ve changed
• Common tips for writing scrapers
  • Fetching a page and setting URLs to absolute paths
  • Getting the current session
  • Common XPath tricks
  • Writing “defensive” scrapers