# Beam Spark Job Server

Beam Spark Job Server is a self-hosted databases app. Containerized Apache Beam Spark job server that executes batch and streaming data-parallel pipelines on an Apache Spark cluster, letting self-hosters run Beam workloads on their own Spark deployments. It is open source and self-hosted, installable on Docker Hub.

Canonical page: https://ourlocalhost.com/app/6171/beam-spark-job-server

## Description

Containerized Apache Beam Spark job server that executes batch and streaming data-parallel pipelines on an Apache Spark cluster, letting self-hosters run Beam workloads on their own Spark deployments.

## Platform availability

Available on 0 of 10 platforms.

## Health score

66/100 (Good), based on 4 of 4 factors.
- Maintained: 100 (last commit 16 days ago)
- Popular: 79 (8,709 GitHub stars)
- Easy to install: 25 (0 install platforms, install notes, Docker image)
- Light to run: 10 (8 GB minimum RAM)

## Links

- Docker Hub: https://hub.docker.com/r/apache/beam_spark_job_server
- GitHub: https://github.com/apache/beam
- Official website: https://beam.apache.org/

## Common questions

### Why might a Python Beam pipeline fail when submitted to Spark from the job server?

It can be caused by a mismatch between the Spark client version in the job server and the Spark version of the cluster you submit to.

### Which Spark master and job-server ports are referenced for local Beam portable runner testing with Spark?

The job-server ports 8097, 8098, and 8099 and the Spark master port 7077 are referenced for local testing.

### Why won’t my Beam job server start after setting up the CLI?

Make sure you saved a valid Beam API token with the CLI. Without the saved token, Beam commands cannot create or control containers, and the job server will not start.

### How are metrics reported when Beam runs on Spark?

The Spark runner uses Spark's built-in metrics system for Beam metrics reporting.

## Alternatives

- BunkerWeb: BunkerWeb is a self-hosted, open-source web application firewall and NGINX-based web server that protects services with secure-by-default s…
- Caddy: Caddy is an extensible web server and reverse proxy that serves Transport Layer Security (TLS) by default, making it simple to secure and r…
- Corvus: Self-hosted reverse proxy and web application firewall with a web console; it provides domain forwarding, free SSL, live traffic monitoring…
- HFS 2 (HTTP File Server): You can use HFS (HTTP File Server) to send and receive files.
- Microsoft IIS: Internet Information Services (IIS) for Windows® Server is a flexible, secure and easy-to-manage Web server for hosting anything on the Web.
- lighttpd: Lighttpd is a secure, lightweight web server that uses memory and CPU more efficiently than comparable servers, with FastCGI, CGI, authenti…
- nginx: Nginx is an HTTP web server, reverse proxy, content cache, load balancer, TCP/UDP proxy, and mail proxy server.

---
Source: Our Local Host (https://ourlocalhost.com/app/6171/beam-spark-job-server), the self-hosted and open-source app directory.
