-
Notifications
You must be signed in to change notification settings - Fork 40
The scala project under examples would cause NoClassDefFoundError #70
Comments
I note the assembly-no-scala.xml file under the project and find the scala library is excluded, but why? |
Hi cfangplus, What Spark version were you using? The examples here are not compatible with the latest Spark-3.0.0-preview yet. If you run into the same issue on Spark 2.x, please provide your environment details and more logs so I could try reproducing this issue. As for the Scala library, I think it's already provided by Spark runtime. |
yea, I noticed that. |
I know this project is developed since June 2019 and at that time the cuda version is 10.1. |
The team is planning to support CUDA 10.2. As for now, you could install both CUDA 10.1 & 10.2 on your server and run these examples with CUDA 10.1: https://stackoverflow.com/questions/41330798/install-multiple-versions-of-cuda-and-cudnn |
That's great. |
I think Joshua answered your question here: rapidsai/cudf#3643 Also as specified by the README.md, this repo provides docs and example applications that demonstrate the RAPIDS.ai GPU-accelerated XGBoost-Spark project. |
@chuanlihao Thank you for your reply. |
Currently cuML only have Python binding. Technically we could apply a similar approach as in XGboost with cuML. Please let us know your use cases for cuML on Spark. |
@anfeng Thanks,I mean we want to accelerate our Spark ML/Graph applications with gpu. |
I followed the steps that is illustrated in the page of
https://github.com/rapidsai/spark-examples/blob/master/getting-started-guides/building-sample-apps/scala.md and built the scala project.
Then I used spark-submit to submit the application to the cluster and I got an exception called
'java.lang.NoClassDefFoundError: scala/Product$class'. It seems that the jar produced by the mvn command does not contain scala library. Please see the attached file for detail.
The text was updated successfully, but these errors were encountered: