theia-spark-operator

Apache Spark operator for Kubernetes, shipped as a Docker image, that makes it easy to specify, run, and track Spark applications on a cluster using…

  • Automation
  • Infrastructure
  • Tools & Utilities
Apache Spark operator for Kubernetes, shipped as a Docker image, that makes it easy to specify, run, and track Spark applications on a cluster using Kubernetes custom resources.

Platform availability

Available on 0 of 10 platforms

  • Unraid (not listed)
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm (not listed)
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

60/100 Good

Maintained
100last commit 15 days ago
Popular
703,157 GitHub stars
Easy to install
100 install platforms, Docker image
Light to run
No data
Security check
201 serious finding, 2 minor, CI, tests, OpenSSF Scorecard 6.7/10

Based on 4 of 5 factors.

GitHub stars
3,157
Open issues
158
Last commit
2026-11-06
Latest release
v2.5.2
License
Apache-2.0
Activity
rising

Checked 15 days ago - source: GitHub (kubeflow/spark-operator)

Resources & Compatibility

ARM (e.g. Raspberry Pi) is not supported.

Checked 54 days ago - source: Docker Hub image tags and published docs

First-install notes

No first-install notes yet.

Alternatives

No alternatives collected yet.

Common questions

What should I check when a Spark driver keeps running with no executors?

The driver can keep running while no executor pods start. Treat the job as stuck on Kubernetes and use the troubleshooting case for that symptom.

Why does the operator sometimes not use all available node resources?

It can schedule only one pod on a node when two could fit. This was reported after executors scale down and then scale up again.

How can I manage Spark pods declaratively with the operator?

The operator manages Spark pods declaratively using YAML custom resources. This is the main tradeoff noted against using spark-submit.

What has a self-hoster reported about day-to-day development with Spark on Kubernetes?

A user testing a Spark operator says it works as advertised. They found development harder than using a dedicated Spark cluster to inspect data.

Answers sourced from stackoverflow.com, stackoverflow.com, www.reddit.com, www.reddit.com

Community

Collected 48 days ago - public community threads