Hadoop

Apache Hadoop is an open-source framework for distributed storage and processing of very large datasets across clusters of commodity servers.

  • Cloud
  • Databases
  • Tools & Utilities
Apache Hadoop is an open-source framework for distributed storage and processing of very large datasets across clusters of commodity servers. It combines HDFS for fault-tolerant file storage with MapReduce for parallel computation, giving data engineers and analysts scalable big-data infrastructure they can run in homelabs or production clusters.

Platform availability

Available on 1 of 10 platforms

  • Unraid (not listed)
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

72/100 Good

Maintained
100last commit 0 days ago
Popular
8415,720 GitHub stars
Easy to install
351 install platform, install notes, Docker image
Light to run
304 GB minimum RAM

Availability: platform listings confirmed today

GitHub stars
15,720
Open issues
332
Last commit
2026-11-21
License
Apache-2.0
Activity
rising

Checked today - source: GitHub (apache/hadoop)

Resources & Compatibility

Minimum requirements: 4 GB RAM, 4 CPU core(s), 70 GB disk. ARM (e.g. Raspberry Pi) is not supported.

Checked 54 days ago - source: Docker Hub image tags and published docs

First-install notes

  • Point the Dynamometer start script to files If you use the manual Dynamometer cluster, run the start script and fill in the paths to the Hadoop package, your config folder, and the HDFS locations it should use. If those paths are missing, the test cluster cannot find the files it needs and will not start. source

Fetched 53 days ago - each note links to its source

Alternatives

No alternatives collected yet.

Common questions

What should I do if Hadoop 3.4.3 does not include the AWS SDK for s3a?

Release 3.4.3 omits the bundle.jar that contains the AWS SDK used by the s3a connector. Download the 2.35.4 bundle artifact from Maven Central and verify its signature.

Which NameNode high-availability options are documented for HDFS?

The HDFS documentation includes guides for NameNode HA with QJM and NameNode HA with NFS Observer.

Why might a manual Dynamometer test cluster fail to start?

The start script must point to the Hadoop package, the config folder, and the HDFS locations to use. If those paths are missing, the test cluster cannot find the files it needs and will not start.

Which configuration files should I check when Hadoop installation errors occur?

Check the path settings in hadoop-env.sh, mapred-site.xml, core-site.xml, and yarn-site.xml.

Answers sourced from hadoop.apache.org, hadoop.apache.org, hadoop.apache.org, www.linkedin.com

Community

Collected 48 days ago - public community threads