FSCrawler

FSCrawler is a file-system crawler that indexes binary documents such as PDF, OpenOffice, and MS Office into Elasticsearch, letting self-hosted users…

  • Databases
  • Productivity
FSCrawler is a file-system crawler that indexes binary documents such as PDF, OpenOffice, and MS Office into Elasticsearch, letting self-hosted users full-text search their document archives.

Platform availability

Available on 1 of 10 platforms

  • Unraid (not listed)
  • TrueNAS
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm (not listed)
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

72/100 Good

Maintained
100last commit 19 days ago
Popular
631,455 GitHub stars
Easy to install
251 install platform, install notes
Light to run
No data
Security check
75no serious findings, 1 minor, CI, tests, OpenSSF Scorecard 5.5/10

Based on 4 of 5 factors.

Availability: platform listings confirmed today, links checked today

GitHub stars
1,455
Open issues
98
Last commit
2026-11-02
Latest release
fscrawler-3.0
License
Apache-2.0
Activity
declining

Checked 19 days ago - source: GitHub (dadoonet/fscrawler)

Resources & Compatibility

Requirements not published yet.

First-install notes

  • Run the one-time setup first Run `bin/fscrawler --setup` before your first crawl; it creates `~/.fscrawler/fscrawler/_settings.yaml`, the job file you edit to tell FSCrawler which files to watch and where to send them. Without this default file, there is no easy place to add your settings. source
  • Point the folder path at your files In `_settings.yaml`, set `fs.url` to the folder you want indexed; if you’re using Docker, use the path inside the container, not the path on your computer. If the path is wrong, FSCrawler will search the wrong place and index nothing or the wrong files. source
  • Add Elasticsearch credentials Set `elasticsearch.urls` to your Elasticsearch address and add `api_key` (preferred) or username/password; this is where FSCrawler stores its indexed file information. Without valid credentials, the crawler cannot save or publish the data it finds. source

Fetched 44 days ago - each note links to its source

Alternatives (12)

  • 4k Video Downloader - Download high-quality videos from various platforms on Windows, macOS, Ubuntu, and Android, supporting playlists, channels, and subtitles.
  • Arroxy - Free open-source YouTube downloader GUI based on yt-dlp for Windows, macOS, and Linux.
  • Cobalt Tools - Ad-free and privacy-focused downloader for videos, music, and media from major social and media platforms, supporting direct sharing link d…
  • Gallery-DL - gallery-dl is a command-line program to download image galleries and collections from several image hosting sites (see https://codeberg.org…
  • Hitomi Downloader - Hitomi Downloader is Desktop application to download images/videos/music/text from Hitomi.la, YouTube, Instagram and other sites, and more…
  • Internet Download Manager - Accelerate your downloads by up to 8 times!
  • JDownloader - JDownloader is a self-hosted JDownloader 2 download manager that runs in a pre-configured dark-mode Selkies web desktop with automatic firs…
  • Media Downloader - Qt/C++ front end to yt-dlp, youtube-dl, gallery-dl, lux, you-get, svtplay-dl, aria2c, wget and safari books.
  • MyInsta - MyInsta is an InstaMod that allows users to have exclusive features such as downloading Reels, Videos, Photos, and No Ads.
  • Stacher - A modern GUI for yt-dlp (and other youtube-dl forks).
  • WFDownloader App - A free batch downloader that can bulk download urls from file/clipboard, generate sequential URLs, download images galleries, wallpapers, m…
  • yt-dlp - Command line program that's a youtube-dl fork that downloads videos from YouTube and over 1000 sites.

Checked 54 days ago - source: AlternativeTo

Common questions

Does FSCrawler keep the Elasticsearch index synchronized when files are added, changed, or removed?

The documentation says it crawls a local file system or mounted drive and indexes new files, updates existing ones, and removes old ones.

Can it crawl files on a remote file system?

Yes, the documentation lists remote file system crawling over SSH or FTP.

Does FSCrawler expose a REST API?

The documentation lists REST among its main features, but the provided excerpt does not detail the endpoints or configuration.

Can it extract text from scanned documents while indexing PDFs?

A Stack Overflow user reports using FSCrawler 2.10 with Tesseract OCR to extract text from scanned documents or images while crawling hundreds of PDF files.

Can it be paired with a simple web frontend for user search?

A Reddit user described planning to use Elasticsearch, FSCrawler, and a simple web frontend for other users, but said they had issues getting FSCrawler to work.

Answers sourced from fscrawler.readthedocs.io, stackoverflow.com, www.reddit.com

Community

Collected 48 days ago - public community threads