Brief Description of the Issue:
When attempting to start the Hadoop services within a Docker container, the Java Home (JAVA_HOME) is not being correctly found, resulting in errors when running the Hadoop startup scripts.
Error Problem:
The following errors are observed when trying to start the Hadoop services:
ERROR: JAVA_HOME is not set and could not be found.
Code Snippet:
in: export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 && $HADOOP_HOME/sbin/start-dfs.sh
out:
root@78a289fdd88e:/# export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 && $HADOOP_HOME/sbin/start-dfs.sh
Starting namenodes on [namenode]
ERROR: JAVA_HOME is not set and could not be found.
Starting datanodes
ERROR: JAVA_HOME is not set and could not be found.
Starting secondary namenodes [78a289fdd88e]
ERROR: JAVA_HOME is not set and could not be found.
My dockerfile:
ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
ENV HADOOP_HOME=/usr/local/hadoop
RUN apt-get update &&
DEBIAN_FRONTEND=noninteractive apt-get install -y
openjdk-8-jdk
ssh
rsync
wget
vim
curl
net-tools
libzip4
libsnappy1v5
libssl-dev
sudo &&
rm -rf /var/lib/apt/lists/*
RUN wget https://archive.apache.org/dist/hadoop/common/hadoop-$HADOOP_VERSION/hadoop-$HADOOP_VERSION.tar.gz -P /tmp &&
tar -xzf /tmp/hadoop-$HADOOP_VERSION.tar.gz -C /usr/local &&
mv /usr/local/hadoop-$HADOOP_VERSION $HADOOP_HOME &&
rm /tmp/hadoop-$HADOOP_VERSION.tar.gz &&
mkdir -p $HADOOP_HOME/logs
RUN mkdir -p /data/dfs/data /data/dfs/name /data/dfs/namesecondary &&
$HADOOP_HOME/bin/hdfs namenode -format
RUN sed -i '1s;^;export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64n;' $HADOOP_HOME/sbin/start-dfs.sh &&
sed -i '1s;^;export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64n;' $HADOOP_HOME/sbin/start-yarn.sh
CMD ["bash", "-c", "export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64; service ssh start; $HADOOP_HOME/sbin/start-dfs.sh; $HADOOP_HOME/sbin/start-yarn.sh; while true; do sleep 1000; done"]
Expected Output:
It is expected that the Hadoop services start correctly after setting the JAVA_HOME. However, the errors persist even after setting the JAVA_HOME.