Spark

Apache Spark is a high-performance engine for large-scale computing tasks, such as data processing, machine learning and real-time data streaming.

  • AI
  • Databases
  • Media Applications
Apache Spark is a high-performance engine for large-scale computing tasks, such as data processing, machine learning and real-time data streaming. It includes APIs for Java, Python, Scala and R.

Platform availability

Available on 1 of 10 platforms

  • Unraid (not listed)
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

78/100 Good

Maintained
100last commit 14 days ago
Popular
9344,429 GitHub stars
Easy to install
150 install platforms, install notes
Light to run
No data

Based on 3 of 4 factors.

GitHub stars
44,429
Open issues
747
Last commit
2026-11-07
License
Apache-2.0
Activity
steady

Checked 14 days ago - source: GitHub (apache/spark)

Resources & Compatibility

Requirements not published yet.

First-install notes

  • Start Spark and check the port Run `npm start` to launch Spark's local test server, then open the address it prints, usually `http://localhost:8080/`. If port 8080 is busy, it may use another port, so check the console log so you don't look at the wrong page. source

Fetched 40 days ago - each note links to its source

Alternatives (2)

  • Zdog - Zdog is a 3D JavaScript engine for and SVG.
  • p5.js - p5.js a JS client-side library for creating graphic and interactive experiences, based on the core principles of Processing.

Checked 40 days ago - source: AlternativeTo

Common questions

How can self-hosters build a basic data lakehouse around Spark?

A basic self-hosted data lakehouse can be set up using Kubernetes, MinIO, and Spark.

What file is required when launching a Spark standalone cluster with the launch scripts?

Create conf/workers in the Spark directory and list the hostnames of all machines where Spark workers should start, one per line.

Can self-hosted Spark process data from Hadoop-based stores?

Spark can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It can run in Hadoop clusters through YARN or Spark's standalone mode.

Are there reliable Docker guides for setting up a standalone Spark server?

Self-hosters looking for a standalone Apache Spark instance on Docker report that comprehensive guides are hard to find, with steps missing.

Answers sourced from github.com, spark.apache.org, www.reddit.com, www.reddit.com

Community

Collected 48 days ago - public community threads