Spark 1.5.2 and Hadoop 2.4 with Mesos 0.24.1 libs for Mesosphere v1.3
1.5K
Download Spark and docker-containers.
Explode Spark tar and cd to directory
Build Spark with make-distribution.sh -Phadoop-2.4
Edit spark/conf/spark-env.sh
Copy core-site.xml, mesos-site.xml, hdfs-site.xml, Dockerfile from docker-containers/spark to spark/dist
Update the xml files if necessary for your settings
From /spark run: cd dist && TAG=1.5.2-hdfs docker build -t eodgooch/spark:$TAG .
FROM mesosphere/mesos:0.24.1-0.2.35.ubuntu1404
MAINTAINER Timothy Chen <[email protected]>
# Set environment variables.
ENV DEBIAN_FRONTEND "noninteractive"
ENV DEBCONF_NONINTERACTIVE_SEEN "true"
# Upgrade package index and install basic commands.
RUN apt-get update && \
apt-get install -y openjdk-7-jdk curl
ENV JAVA_HOME /usr/lib/jvm/java-7-openjdk-amd64
ENV MESOS_NATIVE_JAVA_LIBRARY /usr/local/lib/libmesos.so
ADD . /opt/spark/dist
ADD hdfs-site.xml /etc/hadoop/hdfs-site.xml
ADD core-site.xml /etc/hadoop/core-site.xml
ADD mesos-site.xml /etc/hadoop/mesos-site.xml
RUN ln -sf /usr/lib/libmesos.so /usr/lib/libmesos-0.24.1.so
WORKDIR /opt/spark/dist
Content type
Image
Digest
sha256:ab9acb92d…
Size
572 MB
Last updated
almost 11 years ago
docker pull eodgooch/spark