Skip to content

julien-diener/sparkwebapp

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 

Repository files navigation

A very simple spring web app that use apache spark

The goal is to take the spark/hdfs csvconverter and make it run from a simple spring servlet.

The main trouble come from conflict between the spark and hadoop dependencies, and maybe the spring ones. In the previous stand alone app csvconverter, the dependencies were only used for compilation, but at run time another, global, jar is used: the $SPARK_HOME/lib/spark-assembly-1.1.1-hadoop2.4.0.jar provided by the spark installation.

The solution used here is to replace the maven dependencies included in the war by this spark-assembly jar. See this SO answer for more details.

Author Julien Diener

About

A very simple spring web app that use apache spark

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages