# FSCrawler

FSCrawler is a file-system crawler that indexes binary documents such as PDF, OpenOffice, and MS Office into Elasticsearch, letting self-hosted users full-text search their document archives. It is open source and self-hosted, installable on TrueNAS.

Canonical page: https://ourlocalhost.com/app/4429/fscrawler

## Description

FSCrawler is a file-system crawler that indexes binary documents such as PDF, OpenOffice, and MS Office into Elasticsearch, letting self-hosted users full-text search their document archives.

## Platform availability

Available on 1 of 10 platforms: TrueNAS.

## Health score

72/100 (Good), based on 4 of 5 factors.
- Maintained: 100 (last commit 19 days ago)
- Popular: 63 (1,455 GitHub stars)
- Easy to install: 25 (1 install platform, install notes)
- Security check: 75 (no serious findings, 1 minor, CI, tests, OpenSSF Scorecard 5.5/10)

## Links

- Official website: https://fscrawler.readthedocs.io/
- TrueNAS: https://apps.truenas.com/catalog/fscrawler/
- Documentation: https://fscrawler.readthedocs.io
- GitHub: https://github.com/dadoonet/fscrawler

## Common questions

### Does FSCrawler keep the Elasticsearch index synchronized when files are added, changed, or removed?

The documentation says it crawls a local file system or mounted drive and indexes new files, updates existing ones, and removes old ones.

### Can it crawl files on a remote file system?

Yes, the documentation lists remote file system crawling over SSH or FTP.

### Does FSCrawler expose a REST API?

The documentation lists REST among its main features, but the provided excerpt does not detail the endpoints or configuration.

### Can it extract text from scanned documents while indexing PDFs?

A Stack Overflow user reports using FSCrawler 2.10 with Tesseract OCR to extract text from scanned documents or images while crawling hundreds of PDF files.

### Can it be paired with a simple web frontend for user search?

A Reddit user described planning to use Elasticsearch, FSCrawler, and a simple web frontend for other users, but said they had issues getting FSCrawler to work.

## Alternatives

- 4k Video Downloader: Download high-quality videos from various platforms on Windows, macOS, Ubuntu, and Android, supporting playlists, channels, and subtitles.
- Arroxy: Free open-source YouTube downloader GUI based on yt-dlp for Windows, macOS, and Linux.
- Cobalt Tools: Ad-free and privacy-focused downloader for videos, music, and media from major social and media platforms, supporting direct sharing link d…
- Gallery-DL: gallery-dl is a command-line program to download image galleries and collections from several image hosting sites (see https://codeberg.org…
- Hitomi Downloader: Hitomi Downloader is Desktop application to download images/videos/music/text from Hitomi.la, YouTube, Instagram and other sites, and more…
- Internet Download Manager: Accelerate your downloads by up to 8 times!
- JDownloader: JDownloader is a self-hosted JDownloader 2 download manager that runs in a pre-configured dark-mode Selkies web desktop with automatic firs…
- Media Downloader: Qt/C++ front end to yt-dlp, youtube-dl, gallery-dl, lux, you-get, svtplay-dl, aria2c, wget and safari books.
- MyInsta: MyInsta is an InstaMod that allows users to have exclusive features such as downloading Reels, Videos, Photos, and No Ads.
- Stacher: A modern GUI for yt-dlp (and other youtube-dl forks).
- WFDownloader App: A free batch downloader that can bulk download urls from file/clipboard, generate sequential URLs, download images galleries, wallpapers, m…
- yt-dlp: Command line program that's a youtube-dl fork that downloads videos from YouTube and over 1000 sites.

---
Source: Our Local Host (https://ourlocalhost.com/app/4429/fscrawler), the self-hosted and open-source app directory.
