TEZ-4569: SCATTER_GATHER + BROADCAST hangs on DAG Recovery

[okumin]

Let an AM correctly restore its state and restart tasks.
https://issues.apache.org/jira/browse/TEZ-4569

[tez-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	27m 8s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+0 🆗	mvndep	6m 32s	Maven dependency ordering for branch
+1 💚	mvninstall	12m 39s	master passed
+1 💚	compile	1m 17s	master passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚	compile	1m 12s	master passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+1 💚	checkstyle	1m 30s	master passed
+1 💚	javadoc	1m 0s	master passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚	javadoc	0m 45s	master passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+0 🆗	spotbugs	0m 50s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	2m 39s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 49s	the patch passed
+1 💚	compile	0m 53s	the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚	javac	0m 53s	the patch passed
+1 💚	compile	0m 45s	the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+1 💚	javac	0m 45s	the patch passed
-0 ⚠️	checkstyle	0m 11s	tez-tests: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 20s	the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚	javadoc	0m 19s	the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+1 💚	findbugs	1m 56s	the patch passed
		_ Other Tests _
+1 💚	unit	5m 8s	tez-dag in the patch passed.
+1 💚	unit	42m 27s	tez-tests in the patch passed.
-1 ❌	asflicense	0m 30s	The patch generated 1 ASF License warnings.
		109m 57s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/1/artifact/out/Dockerfile
GITHUB PR	#361
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux 4a42ce80b08c 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / e08d027
Default Java	Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
checkstyle	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/1/artifact/out/diff-checkstyle-tez-tests.txt
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/1/testReport/
asflicense	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/1/artifact/out/patch-asflicense-problems.txt
Max. process+thread count	1233 (vs. ulimit of 5500)
modules	C: tez-dag tez-tests U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/1/console
versions	git=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[tez-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 32s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+0 🆗	mvndep	7m 2s	Maven dependency ordering for branch
+1 💚	mvninstall	16m 6s	master passed
+1 💚	compile	1m 21s	master passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚	compile	1m 12s	master passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+1 💚	checkstyle	1m 33s	master passed
+1 💚	javadoc	1m 4s	master passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚	javadoc	0m 44s	master passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+0 🆗	spotbugs	0m 48s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	2m 49s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 48s	the patch passed
+1 💚	compile	0m 52s	the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚	javac	0m 52s	the patch passed
+1 💚	compile	0m 48s	the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+1 💚	javac	0m 48s	the patch passed
-0 ⚠️	checkstyle	0m 12s	tez-tests: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 20s	the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚	javadoc	0m 20s	the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+1 💚	findbugs	1m 59s	the patch passed
		_ Other Tests _
+1 💚	unit	5m 11s	tez-dag in the patch passed.
+1 💚	unit	41m 25s	tez-tests in the patch passed.
+1 💚	asflicense	0m 31s	The patch does not generate ASF License warnings.
		86m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/2/artifact/out/Dockerfile
GITHUB PR	#361
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux 1a7a0c602b4f 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / e08d027
Default Java	Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
checkstyle	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/2/artifact/out/diff-checkstyle-tez-tests.txt
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/2/testReport/
Max. process+thread count	1162 (vs. ulimit of 5500)
modules	C: tez-dag tez-tests U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/2/console
versions	git=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.