[SPARK-48002][PYTHON][SS][TESTS] Adds sleep before event testing afte…

…r query termination ### What changes were proposed in this pull request? This PR is a followup of apache#46237 that makes to wait 5 secs after the query termination to make sure the events arrive. ### Why are the changes needed? To deflake the test. It's flaky (https://github.com/apache/spark/actions/runs/8873809388/job/24360221027) ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? Manually. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#46275 from HyukjinKwon/SPARK-48002-followup. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
pan3793 · Apr 29, 2024 · eaed585 · eaed585
1 parent ff0751a
commit eaed585
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/python/pyspark/sql/tests/streaming/test_streaming_listener.py b/python/pyspark/sql/tests/streaming/test_streaming_listener.py
@@ -232,6 +232,8 @@ def onQueryTerminated(self, event):
             while q.lastProgress is None or q.lastProgress["batchId"] == 0:
                 q.awaitTermination(0.5)
 
+            time.sleep(5)
+
             self.assertTrue(error_listener.num_rows > 0)
             self.assertTrue(error_listener.num_error_rows > 0)