Spark with Scala: 10. SPARK : LOCAL SETUP

10. SPARK : LOCAL SETUP - Eclipse and Deploy JAR

Env/LAB Setup
##################

Editor - Scala IDE:

Sacle IDE download -> http://scala-ide.org/download/sdk.html
Download for Windows version
Unzip under Spark_Scala/NewDev/scala-SDK-4.7.0-vfinal-2.12-win32.win32.x86_64

Ecliplse :
Spark_Scala/NewDev/scala-SDK-4.7.0-vfinal-2.12-win32.win32.x86_64/eclipse -> click on eclipse and open and give workspace

MAVEN : Integrate with Maven (Dependency manager)

Create folder spark_Scala/Repository
Download Maven : https://maven.apache.org/download.cgi
Unzip under Spark_Scala\Development\apache-maven-3.6.3-src

Create First Project :

Open Eclispse -> New -> Maven Project -> "Practice"

Change Scala Compiler -> RtClk -> Properties -> Scala Compiler -> Latest 2.10.6

Maven settings file change -> provide Repository path where to download all jars -> apache-maven-3.6.3-src -> settings.xml -> then go back to Eclipse -> Window -> Preferences -> Maven -> User Settings -> Browse for Repository path -> Apply and Close

Maven pom.xml change -> Add spark core & spark sql dependencies

Eclipse -> Create "Scala Object" -> "Demo.FruitCount" -> Write code -> Rtclk -> Export -> JAR file -> venkat.jar

FTP -> upload "venkat.jar"

Webconsole -> spark-submit --class Demo.FruitCount Desktop/Jars/venkat.jar