theia-spark-operator
Apache Spark operator for Kubernetes, shipped as a Docker image, that makes it easy to specify, run, and track Spark applications on a cluster using…
Platform availability
Available on 0 of 10 platforms
- Unraid (not listed)
- TrueNAS (not listed)
- Umbrel (not listed)
- ZimaOS (not listed)
- Proxmox (not listed)
- Helm (not listed)
- Mac (not listed)
- Windows (not listed)
- Linux (not listed)
- Any server (not listed)
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
60/100 Good
- Maintained
- 100last commit 15 days ago
- Popular
- 703,157 GitHub stars
- Easy to install
- 100 install platforms, Docker image
- Light to run
- No data
- Security check
- 201 serious finding, 2 minor, CI, tests, OpenSSF Scorecard 6.7/10
Based on 4 of 5 factors.
Security check: 1 serious finding
- Uses certifi 2026.5.20, which has a known critical vulnerability (PyPI)
Automated checks on commit 9012282, 21 Oct 2026. Not an audit.
- GitHub stars
- 3,157
- Open issues
- 158
- Last commit
- 2026-11-06
- Latest release
- v2.5.2
- License
- Apache-2.0
- Activity
- rising
Checked 15 days ago - source: GitHub (kubeflow/spark-operator)
Resources & Compatibility
ARM (e.g. Raspberry Pi) is not supported.
Checked 54 days ago - source: Docker Hub image tags and published docs
First-install notes
No first-install notes yet.
Variants
Alternatives
No alternatives collected yet.
Common questions
What should I check when a Spark driver keeps running with no executors?
The driver can keep running while no executor pods start. Treat the job as stuck on Kubernetes and use the troubleshooting case for that symptom.
Why does the operator sometimes not use all available node resources?
It can schedule only one pod on a node when two could fit. This was reported after executors scale down and then scale up again.
How can I manage Spark pods declaratively with the operator?
The operator manages Spark pods declaratively using YAML custom resources. This is the main tradeoff noted against using spark-submit.
What has a self-hoster reported about day-to-day development with Spark on Kubernetes?
A user testing a Spark operator says it works as advertised. They found development harder than using a dedicated Spark cluster to inspect data.
Answers sourced from stackoverflow.com, stackoverflow.com, www.reddit.com, www.reddit.com
Community
- Spark-operator: infinite spark-pi-driver running with no executors (forum)
- Spark operator vs spark submit - how to choose which one to use? (forum)
- Trending 'spark-operator' questions - Stack Overflow (forum)
Collected 48 days ago - public community threads