Brozzler
Brozzler is a distributed web crawler that drives a real browser (Chrome or Chromium) to fetch pages, extract embedded links, and capture media via yt-dlp,…
Platform availability
Available on 0 of 10 platforms
- Unraid (not listed)
- TrueNAS (not listed)
- Umbrel (not listed)
- ZimaOS (not listed)
- Proxmox (not listed)
- Helm (not listed)
- Mac (not listed)
- Windows (not listed)
- Linux (not listed)
- Any server (not listed)
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
53/100 Fair
- Maintained
- 88last commit 2 days ago
- Popular
- 58824 GitHub stars
- Easy to install
- 250 install platforms, install notes, Docker image
- Light to run
- No data
- Security check
- 54 serious findings, 14 minor, CI, tests, OpenSSF Scorecard 3.6/10
Based on 4 of 5 factors.
Security check: 4 serious findings
- Uses brotlicffi 1.1.0.0, which has a known critical vulnerability (PyPI)
- Uses certifi 2026.1.4, which has a known critical vulnerability (PyPI)
- Uses pillow 12.1.1, which has a known critical vulnerability (PyPI)
- Uses pillow 12.1.1, which has a known critical vulnerability (PyPI)
Automated checks on commit 5ca4efa, 28 Oct 2026. Not an audit.
- GitHub stars
- 824
- Open issues
- 63
- Last commit
- 2026-11-19
- Latest release
- v1.10.0
- License
- Apache-2.0
- Activity
- rising
Checked 2 days ago - source: GitHub (internetarchive/brozzler)
Resources & Compatibility
ARM (e.g. Raspberry Pi) is not supported.
Checked 54 days ago - source: Docker Hub image tags and published docs
First-install notes
- Start RethinkDB before jobs Install Brozzler with the RethinkDB extra and start the RethinkDB (a database service Brozzler uses) before running `brozzler-new-job` or `brozzler-worker`. Brozzler uses it to track jobs; if it is not running, jobs may fail to start. source
- Create a simple job file Make a YAML job file with an id and at least one starting URL under `seeds`, and if using Vagrant set `proxy: localhost:8000`. Then run `brozzler-new-job job1.yml`; without a seed URL, the crawler has nowhere to begin. source
- Use scope to keep it on target Add `scope` rules such as accepts, blocks, and `max_hops` so the crawler only visits pages you intended. This avoids wasting time on unrelated sites, login pages, or other pages you do not want archived. source
- Provide a display for the browser Brozzler drives a real browser, which needs a graphical environment to run; on a headless server you must supply one (e.g. a virtual display), as stated in the README. source
Fetched 42 days ago - each note links to its source
Variants
Alternatives
No alternatives collected yet.
Common questions
Which X11 variant is recommended when running Brozzler on a server?
Xvnc4 is preferred over Xvfb for server use because Chromium on Xvfb did not support screenshots when last tested.
How does Brozzler find a warcprox instance in a clustered deployment?
Run workers with --warcprox-auto so Brozzler looks for a healthy warcprox instance in the RethinkDB service registry. The warcprox instance must point to the same RethinkDB services table.
What installation issue is reported for Brozzler [easy] on Cygwin and Linux?
The issue reports that Brozzler 1.5.18 installs, but Brozzler [easy] does not due to various dependency conflicts.
How can I quickly test Brozzler on one URL?
Use the brozzle-page command-line utility and pass the URL to crawl.
Answers sourced from github.com, github.com
Community
Collected 48 days ago - public community threads