Fix Flaky Tests in /gremlin-core

[qz0610]

Fix 1

org.apache.tinkerpop.gremlin.process.traversal.step.map.ConjoinStepTest.testReturnTypes was found to be flaky using nondex:

[ERROR]   ConjoinStepTest.testReturnTypes:58 expected:<1[0.11.12]> but was:<1[1.12.10]>

This flakiness can be reproduced with command:

mvn -pl gremlin-core edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest=org.apache.tinkerpop.gremlin.process.traversal.step.map.ConjoinStepTest#testReturnTypes -Drat.skip=true

The test was flaky due to the use of an unordered collection (HashSet), that caused assertion failures under different iteration orders. This PR fixed this flakiness by replacing the HashSet with LinkedHashSet

Fix 2

org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.PathProcessorStrategyTest was wound to be flaky for four sets of parameters:

[ERROR]   PathProcessorStrategyTest.doTest:84 __.select(Pop.first,"a","b").by("name").by("age") expected:<[SelectOneStep(first,b,null), TraversalMapStep(value(age))@[b], SelectOneStep(first,a,null), TraversalMapStep(value(name))@[a], SelectStep(last,[a, b])]> but was:<[SelectOneStep(first,a,null), TraversalMapStep(value(name))@[a], SelectOneStep(first,b,null), TraversalMapStep(value(age))@[b], SelectStep(last,[b, a])]>
[ERROR]   PathProcessorStrategyTest.doTest:84 __.select("a","b","c").by("name").by(__.outE().count()) expected:<[SelectOneStep(last,c,null), TraversalMapStep(value(name))@[c], SelectOneStep(last,b,null), TraversalMapStep([VertexStep(OUT,edge), CountGlobalStep])@[b], SelectOneStep(last,a,null), TraversalMapStep(value(name))@[a], SelectStep(last,[a, b, c])]> but was:<[SelectOneStep(last,a,null), TraversalMapStep(value(name))@[a], SelectOneStep(last,c,null), TraversalMapStep(value(name))@[c], SelectOneStep(last,b,null), TraversalMapStep([VertexStep(OUT,edge), CountGlobalStep])@[b], SelectStep(last,[b, c, a])]>
[ERROR]   PathProcessorStrategyTest.doTest:84 __.select("a","b").by().by("age") expected:<[SelectOneStep(last,b,null), TraversalMapStep(value(age))@[b], SelectOneStep(last,a,null), TraversalMapStep(identity)@[a], SelectStep(last,[a, b])]> but was:<[SelectOneStep(last,a,null), TraversalMapStep(identity)@[a], SelectOneStep(last,b,null), TraversalMapStep(value(age))@[b], SelectStep(last,[b, a])]>
[ERROR] org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.PathProcessorStrategyTest.doTest[[SelectStep(last,[a, b],[value(name), value(age)])]](org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.PathProcessorStrategyTest)
[ERROR]   Run 1: PathProcessorStrategyTest.doTest:84 __.select("a","b").by("name").by("age") expected:<[SelectOneStep(last,b,null), TraversalMapStep(value(age))@[b], SelectOneStep(last,a,null), TraversalMapStep(value(name))@[a], SelectStep(last,[a, b])]> but was:<[SelectOneStep(last,a,null), TraversalMapStep(value(name))@[a], SelectOneStep(last,b,null), TraversalMapStep(value(age))@[b], SelectStep(last,[b, a])]>
[ERROR]   Run 2: PathProcessorStrategyTest.doTest:84 __.select(Pop.last,"a","b").by("name").by("age") expected:<[SelectOneStep(last,b,null), TraversalMapStep(value(age))@[b], SelectOneStep(last,a,null), TraversalMapStep(value(name))@[a], SelectStep(last,[a, b])]> but was:<[SelectOneStep(last,a,null), TraversalMapStep(value(name))@[a], SelectOneStep(last,b,null), TraversalMapStep(value(age))@[b], SelectStep(last,[b, a])]>

The flakiness can be reproduced with command:

mvn -pl gremlin-core edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest=org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.PathProcessorStrategyTest  -DnondexMode=ONE -Drat.skip=true

The cause of this flakiness is that when processing select("a","b").by(...).by(...), the order of keys is not guaranteed due to use of HashMap in selectStep.getByTraversals(), resulting in assertion failures non-deterministically. This PR fixed this flakiness by replacing the HashMap with LinkedHashMap.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.43%. Comparing base (9b46b67) to head (0262b38).
Report is 279 commits behind head on 3.7-dev.

Additional details and impacted files

@@              Coverage Diff              @@
##             3.7-dev    #2943      +/-   ##
=============================================
+ Coverage      76.14%   76.43%   +0.28%     
- Complexity     13152    13226      +74     
=============================================
  Files           1084     1060      -24     
  Lines          65160    61491    -3669     
  Branches        7285     7336      +51     
=============================================
- Hits           49616    46998    -2618     
+ Misses         12839    11973     -866     
+ Partials        2705     2520     -185     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Cole-Greer]

Hi @qz0610, thanks for once again identifying these flaky tests and improving them!

VOTE +1 (merging as CTR)