Beam_flink1.8_job_server

Apache Beam's Flink job server is a container that translates Apache Beam pipelines into jobs the Apache Flink engine can execute.

  • Databases
  • Tools & Utilities
Apache Beam's Flink job server is a container that translates Apache Beam pipelines into jobs the Apache Flink engine can execute. It runs batch and streaming data-parallel processing pipelines (ETL, analytics, data movement) on a self-hosted Flink 1.8 cluster, for developers who want Beam's unified pipeline model with Flink's distributed execution.

Platform availability

Available on 0 of 10 platforms

  • Unraid (not listed)
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm (not listed)
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

76/100 Good

Maintained
100last commit 15 days ago
Popular
798,711 GitHub stars
Easy to install
250 install platforms, install notes, Docker image
Light to run
No data

Based on 3 of 4 factors.

GitHub stars
8,711
Open issues
3875
Last commit
2026-11-06
Latest release
v2.77.0
License
Apache-2.0
Activity
rising

Checked 15 days ago - source: GitHub (apache/beam)

Resources & Compatibility

ARM (e.g. Raspberry Pi) is not supported.

Checked 54 days ago - source: Docker Hub image tags and published docs

First-install notes

  • Install and sign in to Beam CLI Install the Beam CLI (the command-line tool that controls Beam containers) by running uv tool install beam-client, where uv is a Python package installer, then run beam configure default --token YOUR_TOKEN to save your login. Without that saved token, Beam commands will not be accepted. source
  • Create your local Beam context Run ./beta9 and answer the prompts for context name, gateway host (where the local service runs), gateway port (default 1993, the network port it listens on), and token. This saves a connection profile; without it, the CLI does not know where your local Beam server is or how to authenticate. source
  • Expose the port your app uses When defining a Beam Pod (the container that runs your app), include the port it listens on in the ports list; the example uses ports=[8000]. If that port is not exposed, the container may start but nothing outside it can reach the job server. source

Fetched 53 days ago - each note links to its source

Alternatives

No alternatives collected yet.

Common questions

How do I point a Beam Flink job server at a separate Flink cluster?

Start the job server container with --net=host and pass the Flink REST endpoint with --flink-master, such as --flink-master=localhost:8081. The reported workaround starts successfully when using that endpoint flag.

Why can't anything outside a Beam Pod reach the job server even if the container starts?

Expose the port the app listens on in the Beam Pod definition, for example ports=[8000]. If that port is not in the ports list, the container may start but outside clients cannot reach the job server.

Why does an Apache Beam pipeline with environment type EXTERNAL fail against the Flink job server?

The job can fail if the external service does not point into something running inside the Flink task manager. The external service endpoint needs to be reachable from inside the task manager.

Do the portable Flink runner instructions work well for Go?

An issue reports that it is difficult to use the Portable Flink Runner with Go and asks for updated instructions. It also asks that the instructions be tested on newer Flink versions.

Answers sourced from docs.beam.cloud, github.com, stackoverflow.com, www.reddit.com

Community

Collected 48 days ago - public community threads