Open-source browser automation, scraping and agent-driven browsing

Browser automation moved from testing into agent infrastructure: an agent that can open a page, read it and act on it needs a browser it can drive reliably and cheaply.

Every website is an API if you are willing to drive a browser, and in 2026 that stopped being a workaround and became a design choice — agents interact with systems that will never publish an interface, and the browser is the interface.

Cost is what separates the projects here. A full Chromium per session is fine for ten tasks and ruinous for ten thousand, so several entries below are attempts to strip the browser down to what an agent actually needs. The rest handle the parts that break: sessions, anti-bot friction, and pages that render nothing without JavaScript.

Projects in this topic

23 analysed projects match this topic.

Web Development

Developer-first web scraping layer for go

Skip weeks of custom work by starting from a working web scraping foundation.

web scrapinggodata extraction
Beginner · Developer library
TOOL-83701View details

Questions people ask

Is scraping legal?
It depends on the data, the jurisdiction and the site's terms — public factual data is treated very differently from personal data or content behind a login. Read the terms and the applicable data-protection rules before building a product on it; this catalog describes what tools do, not whether your use of them is lawful.
Do I need a full browser?
Only when the page needs JavaScript to produce its content. If the HTML already contains what you want, an HTTP client is an order of magnitude cheaper and far more reliable.
What breaks these pipelines most often?
Layout changes and session handling, not blocking. Selectors that describe intent rather than position survive redesigns; selectors copied from devtools do not.

Related collections

All collections

Other topics

Updated: 2026-08-17