Command-line toolCurrent v0.3.1

Web Scraper Toolkit

A scraper and MCP server for turning difficult web pages into Markdown, JSON, and other useful outputs.

  • Command line
  • MCP server
  • PyPI
See the install path View source on GitHub
Works on
  • Python
  • AI coding agents
Latest
v0.3.1
Released
Mar 6, 2026
Language
Python
Current files
Install path
Web Scraper Toolkit browser extraction workflow cover.
Start here

Install the current package

pip install web-scraper-toolkit && playwright install

The package provides the CLI, Python API, and MCP server. Install the browser runtime needed by the workflow you are authorized to run.

Read the toolkit READMEOpen the current release on GitHub
The point of it

What it does

Engineers, analysts, and agent builders who need repeatable browser extraction rather than a one-off copy-and-paste scrape.

The same project can run a URL through a CLI, Python API, or MCP server while keeping output formats, browser behavior, and diagnostics explicit.

  • Scrape one page or a batch of URLs into Markdown, JSON, text, PDF, or another supported output.
  • Run the MCP server when an agent needs controlled browser navigation and extraction tools.
  • Use Playwright and native-browser fallback diagnostics when a site blocks the first route.
  • Respect site terms, access controls, privacy expectations, and robots guidance for every target.

Keep in mind: Only automate pages and data you are authorized to access. Browser fallback and anti-bot diagnostics do not change a site's terms, access controls, or privacy expectations.

Open the project README