beam_flink1.10_job_server

Apache Beam's Flink job server is a coordinator that schedules and dispatches portable Apache Beam pipelines to a distributed Apache Flink cluster.

  • Automation
  • Databases
  • Tools & Utilities
Apache Beam's Flink job server is a coordinator that schedules and dispatches portable Apache Beam pipelines to a distributed Apache Flink cluster. It organises the execution of large-scale batch and streaming data jobs so they run reliably across multiple Flink workers. For data engineers deploying Beam pipelines on their own Flink deployment.

Platform availability

Available on 1 of 10 platforms

  • Unraid
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm (not listed)
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

69/100 Good

Maintained
100last commit 2 days ago
Popular
798,722 GitHub stars
Easy to install
250 install platforms, install notes, Docker image
Light to run
304 GB minimum RAM
GitHub stars
8,722
Open issues
3705
Last commit
2026-11-19
Latest release
v2.78.0
License
Apache-2.0
Activity
rising

Checked 2 days ago - source: GitHub (apache/beam)

Resources & Compatibility

Minimum requirements: 4 GB RAM, 30 GB disk. ARM (e.g. Raspberry Pi) is not supported.

Checked 54 days ago - source: Docker Hub image tags and published docs

First-install notes

  • Add your Beam API token before running Install the Beam command-line tool and run the configure command with your API token, which is a private code from your Beam account. It stores your credentials on your computer so Beam can create sandboxes (temporary workspaces where your app runs); without it, later commands fail because Beam does not know who you are. source
  • Set up the local gateway before starting If you are self-hosting, run the interactive Beta9 setup menu and enter a setup name, gateway host (where the local gateway is running), port (the network number it listens on), and token. The gateway is the local front door for Beam requests; if these values are missing or wrong, jobs may not start or connect. source
  • Install the Beam client for your language If your project is Python or TypeScript, install the matching Beam client package so your code can create sandboxes (temporary workspaces). Without the right package, the program stops because it cannot find the Beam library. source

Fetched 53 days ago - each note links to its source

Alternatives

No alternatives collected yet.

Common questions

How do I point beam_flink1.10_job_server at a separate Flink cluster?

Start the job server container with host networking and pass the Flink REST endpoint using --flink-master, for example localhost:8081.

Why must my local Beam SDK version match the job server image?

The locally installed Apache Beam version must match the Beam version used by the SDK and job server. For example, a Python 2.41 Apache Beam install should use the corresponding 2.41 Flink job server image.

How are Flink 2 and later job server images tagged?

For Flink 2 and later, the images share one repository and include both the Beam and Flink versions in the tag, such as beam_flink_job_server:2.75.0-flink2.0.

What known issue affects the portable Flink runner instructions?

A GitHub issue reports that using the portable Flink runner with Go is difficult and requests updated instructions. It also asks for testing on newer Flink versions such as 1.16 or 1.17.

Answers sourced from beam.apache.org, github.com, stackoverflow.com, www.cheeyeo.xyz

Community

Collected 48 days ago - public community threads