# theia-spark-operator

theia-spark-operator is a self-hosted automation app. Apache Spark operator for Kubernetes, shipped as a Docker image, that makes it easy to specify, run, and track Spark applications on a cluster using Kubernetes custom resources. It is open source and self-hosted, installable on Docker Hub.

Canonical page: https://ourlocalhost.com/app/5409/theia-spark-operator

## Description

Apache Spark operator for Kubernetes, shipped as a Docker image, that makes it easy to specify, run, and track Spark applications on a cluster using Kubernetes custom resources.

## Platform availability

Available on 0 of 10 platforms.

## Health score

60/100 (Good), based on 4 of 5 factors.
- Maintained: 100 (last commit 15 days ago)
- Popular: 70 (3,157 GitHub stars)
- Easy to install: 10 (0 install platforms, Docker image)
- Security check: 20 (1 serious finding, 2 minor, CI, tests, OpenSSF Scorecard 6.7/10)

## Security check

1 serious finding: Uses certifi 2026.5.20, which has a known critical vulnerability.
Automated checks on commit 9012282, 21 Oct 2026. Not an audit.

## Links

- Docker Hub: https://hub.docker.com/r/antrea/theia-spark-operator
- Official website: https://spark.kubeflow.org/en/latest/
- GitHub: https://github.com/kubeflow/spark-operator

## Common questions

### What should I check when a Spark driver keeps running with no executors?

The driver can keep running while no executor pods start. Treat the job as stuck on Kubernetes and use the troubleshooting case for that symptom.

### Why does the operator sometimes not use all available node resources?

It can schedule only one pod on a node when two could fit. This was reported after executors scale down and then scale up again.

### How can I manage Spark pods declaratively with the operator?

The operator manages Spark pods declaratively using YAML custom resources. This is the main tradeoff noted against using spark-submit.

### What has a self-hoster reported about day-to-day development with Spark on Kubernetes?

A user testing a Spark operator says it works as advertised. They found development harder than using a dedicated Spark cluster to inspect data.

---
Source: Our Local Host (https://ourlocalhost.com/app/5409/theia-spark-operator), the self-hosted and open-source app directory.
