# Hadoop

Hadoop is a self-hosted cloud app. Apache Hadoop is an open-source framework for distributed storage and processing of very large datasets across clusters of commodity servers. It is open source and self-hosted, installable on Docker Hub and Kubernetes (Helm).

Canonical page: https://ourlocalhost.com/app/5871/hadoop

## Description

Apache Hadoop is an open-source framework for distributed storage and processing of very large datasets across clusters of commodity servers. It combines HDFS for fault-tolerant file storage with MapReduce for parallel computation, giving data engineers and analysts scalable big-data infrastructure they can run in homelabs or production clusters.

## Platform availability

Available on 1 of 10 platforms: Helm.

## Health score

72/100 (Good), based on 4 of 4 factors.
- Maintained: 100 (last commit 0 days ago)
- Popular: 84 (15,720 GitHub stars)
- Easy to install: 35 (1 install platform, install notes, Docker image)
- Light to run: 30 (4 GB minimum RAM)

## Links

- Docker Hub: https://hub.docker.com/r/apache/hadoop
- Kubernetes (Helm): https://artifacthub.io/packages/helm/apache-hadoop-helm/hadoop
- GitHub: https://github.com/apache/hadoop
- Official website: https://hadoop.apache.org/
- Documentation: https://hadoop.apache.org/docs/stable

## Common questions

### What should I do if Hadoop 3.4.3 does not include the AWS SDK for s3a?

Release 3.4.3 omits the bundle.jar that contains the AWS SDK used by the s3a connector. Download the 2.35.4 bundle artifact from Maven Central and verify its signature.

### Which NameNode high-availability options are documented for HDFS?

The HDFS documentation includes guides for NameNode HA with QJM and NameNode HA with NFS Observer.

### Why might a manual Dynamometer test cluster fail to start?

The start script must point to the Hadoop package, the config folder, and the HDFS locations to use. If those paths are missing, the test cluster cannot find the files it needs and will not start.

### Which configuration files should I check when Hadoop installation errors occur?

Check the path settings in hadoop-env.sh, mapred-site.xml, core-site.xml, and yarn-site.xml.

---
Source: Our Local Host (https://ourlocalhost.com/app/5871/hadoop), the self-hosted and open-source app directory.
