Crawl Log Viewer
Crawl Log Viewer is a standalone web service that retrieves and filters crawl log streams stored as Apache Kafka topics, showing crawler activity with…
Platform availability
Available on 0 of 10 platforms
- Unraid (not listed)
- TrueNAS (not listed)
- Umbrel (not listed)
- ZimaOS (not listed)
- Proxmox (not listed)
- Helm (not listed)
- Mac (not listed)
- Windows (not listed)
- Linux (not listed)
- Any server (not listed)
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
12/100 Low
- Maintained
- 0last commit 1026 days ago
- Popular
- 00 GitHub stars
- Easy to install
- 250 install platforms, install notes, Docker image
- Light to run
- 502 GB minimum RAM
- GitHub stars
- 0
- Open issues
- 0
- Last commit
- 2024-01-30
- License
- Apache-2.0
- Activity
- stale
Checked 18 days ago - source: GitHub (ldbiz/crawl-log-viewer)
Resources & Compatibility
Minimum requirements: 2 GB RAM, 2 CPU core(s). ARM (e.g. Raspberry Pi) is not supported.
Checked 54 days ago - source: Docker Hub image tags and published docs
First-install notes
- Install the Python pieces it needs Make sure Python and pip (the Python package installer) are available, then run `python -m pip install -r requirements.txt`. Without these packages, the app’s commands can fail because they can’t import the code they need. source
- Fill in the MCP client settings In your MCP (Model Context Protocol) client config, set the Python executable, the path to `server.py`, and `SEARXNG_URL` if you’re using SearXNG (a search engine). If these paths or URLs are wrong, the client can’t start the tool or won’t know where search requests should go. source
- Start it from the right folder After setup, start the server from the crawl folder with `crawl-mcp` or `python server.py`. If you use the wrong folder, the MCP client may not be able to reach the server. source
- Configure topics.json for your Kafka topics Create a topics.json configuration file pointing the service at your Kafka brokers and the crawl log topics to inspect; the view defaults to the previous day's activity from the first topic in the list. source
Fetched 43 days ago - each note links to its source
Variants
Alternatives
No alternatives collected yet.
Common questions
Can a crawl log be used to diagnose crawler problems?
Yes. A crawl log can show the status of crawled content and help diagnose problems.
Can it work alongside existing self-hosted monitoring tools?
A self-hoster asked for a lightweight, Docker-friendly web log viewer to complement tools such as Netdata, Dozzle, and Glances.
Can it view non-Docker log files without a full log aggregator?
A self-hoster asked for a Dozzle-like clean web interface for non-Docker log files, without using a full log aggregator like Loki.
What do self-hosters want in a home server log monitoring setup?
A self-hoster wanted a simple, maintainable log monitoring solution for a home server.
Answers sourced from support.microsoft.com, www.reddit.com, www.reddit.com, www.reddit.com
Community
- Follow-Up: Web-Based log viewer : r/selfhosted - Reddit (reddit)
- Log viewer like dazzle for non docker logs : r/selfhosted - Reddit (reddit)
- Monitoring log files : r/selfhosted - Reddit (reddit)
Collected 48 days ago - public community threads