Quick Expense Manger. Your free expense manager. Lots of features. The application is also ad free.

Apache Spark Installation in 6 Steps

Posted on Nov. 10, 2017
apache-spark-installation-6-steps

I have installed Apache Spark on Ubuntu 14.04. I have gone through many hardships to install this as the installation documentation is not good. Here I am documenting each and every step required to successfully install Apache Spark on Ubuntu.

#1. Run the following command to check if Java is Installed

java -version from terminal. If java is not installed, run the command sudo apt-get install default-jdk

#2. Run the following command to download scala

sudo wget www.scala-lang.org/files/archive/scala-2.11.7.deb

If you are running behind a proxy then run the following command to download scala

sudo wget -e use_proxy=yes -e http_proxy=myproxy.com:myPort www.scala-lang.org/files/archive/scala-2.11.7.deb

If you don't know your proxy settings, run the following command to check your proxy settings.

command set | grep -i proxy

Once you have downloaded scala, install it using the following command.

sudo dpkg -i scala-2.11.7.deb

Once scala is installed verify the installation using the following command

scala -version

#3. Download maven using the following command

sudo wget http://mirror.fibergrid.in/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz

If you are running behind a proxy, use the proxy settings as mentioned in point (2)

Once you have downloaded maven, extract the .tar.gz file using the following command

tar xvf apache-maven-3.3.9-bin.tar.gz

After the command is run, the folder apache-maven-3.3.9-bin will be created. Move this directory to /usr/local directory using the following command.

sudo mv apache-maven-3.3.9-bin /usr/local/

Put the binary in the PATH using the following command

export PATH=$PATH:/usr/local/apache-maven-3.3.9-bin/bin

#4. Download Apache Spark 1.6.1 using the following command

sudo wget http://a.mbbsindia.com/spark/spark-1.6.1/spark-1.6.1.tgz

Once you have downloaded Spark, extract the .tar.gz file using the following command

tar xvf spark-1.6.1.tgz

After the command is run, the folder spark-1.6.1 will be created. Move this directory to /usr/local directory using the following command.

sudo mv spark-1.6.1 /usr/local/

Put the binary in the PATH using the following command

export PATH=$PATH:/usr/local/spark-1.6.1/bin

#5. Go to the directory /usr/local/spark-1.6.1 using the following command

cd /usr/local/spark-1.6.1

Now, run the following command to install Apache Spark

mvn -DskipTests clean package

If you are running behind a proxy, then use the following command instead.

mvn -DproxySet=true -DproxyHost=myproxy.com -DproxyPort=myPort -DskipTests clean package

#6. After Apache Spark is installed, verify the installation by running the following command

spark-shell

You should see the following screen





Sharing is Caring!

Quick Expense Manger. Your free expense manager. Lots of features. The application is also ad free.

GET FREE UPDATES


RECOMMENDED POSTS FOR YOU


profile image

Kaushik Baruah


ABOUT

My name is Kaushik Baruah and I am the chief blogger on this Blog and here I like to share my experience as software engineer and research engineer with my online readers. I will try to focus on career planning, latest emerging technologies and tutorials on various computer science subjects. You can follow me on Twitter, Facebook and Google+

GET FREE UPDATES

POPULAR POSTS

Copyright © 2016
About Us

My name is Kaushik Baruah and I am the chief blogger on this Blog and here I like to share my experience as software engineer and research engineer with my online readers. I will try to focus on career planning, latest emerging technologies and tutorials on various computer science subjects.

Get Free Updates