# Spark Kubernetes Operator

Spark Kubernetes Operator is a self-hosted automation app. A Kubernetes operator from the Apache Software Foundation that extends cluster resource management to deploy and manage Apache Spark applications via the Operator pattern, giving data engineers and cluster operators a... It is open source and self-hosted, installable on Kubernetes (Helm).

Canonical page: https://ourlocalhost.com/app/4858/spark-kubernetes-operator

## Description

A Kubernetes operator from the Apache Software Foundation that extends cluster resource management to deploy and manage Apache Spark applications via the Operator pattern, giving data engineers and cluster operators a native Kubernetes workflow for Spark workloads.

## Platform availability

Available on 1 of 10 platforms: Helm.

## Health score

68/100 (Good), based on 5 of 5 factors.
- Maintained: 100 (last commit 14 days ago)
- Popular: 50 (327 GitHub stars)
- Easy to install: 25 (1 install platform, install notes)
- Light to run: 30 (4 GB minimum RAM)
- Security check: 95 (no serious findings, CI, tests)

## Links

- GitHub: https://github.com/apache/spark-kubernetes-operator
- Official website: https://apache.github.io/spark-kubernetes-operator/
- Kubernetes (Helm): https://artifacthub.io/packages/helm/spark-kubernetes-operator/spark-kubernetes-operator
- Documentation: https://spark.apache.org/docs/latest/running-on-kubernetes.html

## Common questions

### Is the operator suitable for interactive Spark clusters or notebooks?

Community reports say it is mainly good for batch jobs. Working with the cluster interactively seems to be a problem.

### Why might exec pods not use all available node capacity?

The operator may not utilize node resources completely, spinning only one pod on a node when two could fit. After execs scale down and up again, pods can end up on separate nodes.

### What can cause a Spark job to stop recovering after Kubernetes kills a pod?

Kubernetes can kill processing pods that exceed their memory limits. One report says that, once Kubernetes killed the pod, Spark could not recover.

## Alternatives

- Zdog: Zdog is a 3D JavaScript engine for and SVG.
- p5.js: p5.js a JS client-side library for creating graphic and interactive experiences, based on the core principles of Processing.
- spark: Apache Spark is a high-performance engine for large-scale computing tasks, such as data processing, machine learning and real-time data str…

---
Source: Our Local Host (https://ourlocalhost.com/app/4858/spark-kubernetes-operator), the self-hosted and open-source app directory.
