Initial GHA for our purposes #20
build_main.yml
on: push
Run
/
Check changes
40s
Run
/
Breaking change detection with Buf (branch-3.5)
54s
Run
/
Run TPC-DS queries with SF=1
1h 41m
Run
/
Run Docker integration tests
28m 58s
Run
/
Run Spark on Kubernetes Integration test
1h 26m
Matrix: Run / build
Matrix: Run / java-other-versions
Run
/
Build modules: sparkr
24m 32s
Run
/
Linters, licenses, dependencies and documentation generation
1h 23m
Matrix: Run / pyspark
Annotations
11 errors and 1 warning
Run / Build modules: streaming, sql-kafka-0-10, streaming-kafka-0-10, yarn, kubernetes, hadoop-cloud, spark-ganglia-lgpl, connect, protobuf
Process completed with exit code 18.
|
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-fcd8d58ba05636ea-exec-1".
|
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-5fefb68ba05774af-exec-1".
|
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-ed431a8ba05ba9c9-exec-1".
|
Run / Run Spark on Kubernetes Integration test
Status(apiVersion=v1, code=404, details=StatusDetails(causes=[], group=null, kind=pods, name=spark-test-app-13ae4353e2144e50a8df0b3551b56663-driver, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=pods "spark-test-app-13ae4353e2144e50a8df0b3551b56663-driver" not found, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=NotFound, status=Failure, additionalProperties={})..
|
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-3f868f8ba0718af4-exec-1".
|
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-eaf3438ba072bd91-exec-1".
|
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-4ce9ae8ba076f22d-exec-1".
|
Run / Run Spark on Kubernetes Integration test
Status(apiVersion=v1, code=404, details=StatusDetails(causes=[], group=null, kind=pods, name=spark-test-app-4cf70b4d0f064bdc9349106f3595909b-driver, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=pods "spark-test-app-4cf70b4d0f064bdc9349106f3595909b-driver" not found, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=NotFound, status=Failure, additionalProperties={})..
|
SparkConnectServiceE2ESuite.ReleaseSession for different user_id with same session_id do not affect each other:
SparkConnectServiceE2ESuite#L136
org.apache.spark.SparkException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext.
This stopped SparkContext was created at:
org.apache.spark.sql.connect.service.SparkConnectSessionHolderSuite.beforeAll(SparkConnectSessionHolderSuite.scala:37)
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.base/java.lang.Thread.run(Thread.java:840)
The currently active SparkContext was created at:
org.apache.spark.sql.connect.service.SparkConnectServiceE2ESuite.beforeAll(SparkConnectServiceE2ESuite.scala:27)
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.base/java.lang.Thread.run(Thread.java:840)
|
SparkConnectServiceE2ESuite.ReleaseSession for different session_id with same user_id do not affect each other:
SparkConnectServiceE2ESuite#L142
org.apache.spark.SparkException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext.
This stopped SparkContext was created at:
org.apache.spark.sql.connect.service.SparkConnectSessionHolderSuite.beforeAll(SparkConnectSessionHolderSuite.scala:37)
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.base/java.lang.Thread.run(Thread.java:840)
The currently active SparkContext was created at:
org.apache.spark.sql.connect.service.SparkConnectServiceE2ESuite.beforeAll(SparkConnectServiceE2ESuite.scala:27)
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.base/java.lang.Thread.run(Thread.java:840)
|
Run / Build modules: pyspark-errors
No files were found with the provided path: **/target/test-reports/*.xml. No artifacts will be uploaded.
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
site
Expired
|
60 MB |
|
test-results-api, catalyst, hive-thriftserver--17-hadoop3-hive2.3
Expired
|
2.8 MB |
|
test-results-core, unsafe, kvstore, avro, utils, network-common, network-shuffle, repl, launcher, examples, sketch, graphx--17-hadoop3-hive2.3
Expired
|
2.6 MB |
|
test-results-docker-integration--17-hadoop3-hive2.3
Expired
|
119 KB |
|
test-results-hive-- other tests-17-hadoop3-hive2.3
Expired
|
911 KB |
|
test-results-hive-- slow tests-17-hadoop3-hive2.3
Expired
|
853 KB |
|
test-results-mllib-local,mllib--17-hadoop3-hive2.3
Expired
|
1.36 MB |
|
test-results-pyspark-connect--17-hadoop3-hive2.3
Expired
|
411 KB |
|
test-results-pyspark-core, pyspark-streaming--17-hadoop3-hive2.3
Expired
|
79.8 KB |
|
test-results-pyspark-mllib, pyspark-ml, pyspark-ml-connect--17-hadoop3-hive2.3
Expired
|
1.09 MB |
|
test-results-pyspark-pandas--17-hadoop3-hive2.3
Expired
|
1.46 MB |
|
test-results-pyspark-pandas-connect-part0--17-hadoop3-hive2.3
Expired
|
1.32 MB |
|
test-results-pyspark-pandas-connect-part1--17-hadoop3-hive2.3
Expired
|
1.42 MB |
|
test-results-pyspark-pandas-connect-part2--17-hadoop3-hive2.3
Expired
|
953 KB |
|
test-results-pyspark-pandas-connect-part3--17-hadoop3-hive2.3
Expired
|
530 KB |
|
test-results-pyspark-pandas-slow--17-hadoop3-hive2.3
Expired
|
2.86 MB |
|
test-results-pyspark-sql, pyspark-resource, pyspark-testing--17-hadoop3-hive2.3
Expired
|
403 KB |
|
test-results-sparkr--17-hadoop3-hive2.3
Expired
|
280 KB |
|
test-results-sql-- extended tests-17-hadoop3-hive2.3
Expired
|
2.97 MB |
|
test-results-sql-- other tests-17-hadoop3-hive2.3
Expired
|
4.26 MB |
|
test-results-sql-- slow tests-17-hadoop3-hive2.3
Expired
|
2.77 MB |
|
test-results-streaming, sql-kafka-0-10, streaming-kafka-0-10, yarn, kubernetes, hadoop-cloud, spark-ganglia-lgpl, connect, protobuf--17-hadoop3-hive2.3
Expired
|
521 KB |
|
test-results-tpcds--17-hadoop3-hive2.3
Expired
|
21.9 KB |
|
unit-tests-log-streaming, sql-kafka-0-10, streaming-kafka-0-10, yarn, kubernetes, hadoop-cloud, spark-ganglia-lgpl, connect, protobuf--17-hadoop3-hive2.3
Expired
|
333 MB |
|