Skip to content

Commit

Permalink
Modify spark.md
Browse files Browse the repository at this point in the history
  • Loading branch information
ChengJie1053 committed Dec 14, 2023
1 parent 090e68a commit 845e383
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 4 deletions.
8 changes: 6 additions & 2 deletions docs/engine-usage/spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,11 +184,15 @@ Upload the jar package and configuration
```shell
# Upload the jar package under the lib of the linkis spark engine (modify the following parameters according to your actual installation directory)
cd /appcom/Install/linkis/lib/linkis-engineconn-plugins/spark/dist/3.2.1/lib
hdfs dfs -put *.jar hdfs:///spark/cluster/
hdfs dfs -put *.jar hdfs:///spark/cluster

# Upload the linkis configuration file (modify the following parameters according to your actual installation directory)
cd /appcom/Install/linkis/conf
hdfs dfs -put * hdfs:///spark/cluster/
hdfs dfs -put * hdfs:///spark/cluster

# Upload hive-site.xml (modify the following parameters according to your actual installation directory)
cd $HIVE_CONF_DIR
hdfs dfs -put hive-site.xml hdfs:///spark/cluster
```
Can pass `linkis.spark.yarn.cluster.jars`parameters to modify`hdfs:///spark/cluster`

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -182,11 +182,15 @@ Token-User: linkis
```shell
# 上传linkis spark引擎的lib下的jar包 (根据您的实际安装目录修改以下参数)
cd /appcom/Install/linkis/lib/linkis-engineconn-plugins/spark/dist/3.2.1/lib
hdfs dfs -put *.jar hdfs:///spark/cluster/
hdfs dfs -put *.jar hdfs:///spark/cluster

# 上传linkis 配置文件 (根据您的实际安装目录修改以下参数)
cd /appcom/Install/linkis/conf
hdfs dfs -put * hdfs:///spark/cluster/
hdfs dfs -put * hdfs:///spark/cluster

# 上传hive-site.xml (根据您的实际安装目录修改以下参数)
cd $HIVE_CONF_DIR
hdfs dfs -put hive-site.xml hdfs:///spark/cluster
```
可以通过`linkis.spark.yarn.cluster.jars`参数来修改`hdfs:///spark/cluster`

Expand Down

0 comments on commit 845e383

Please sign in to comment.