Skip to content

Commit

Permalink
Tests: disable batching in gold tests on Android
Browse files Browse the repository at this point in the history
setRequestedOrientation from a previous trace appears to affect pixels.
We often see this on CI when a new trace is added which reshuffles
traces.

For example, in this sequence the first trace sets landscape orientation
and the second one goes back to portrait:

    angle_trace_tests --gtest_filter=TraceTest.dota_underlords:TraceTest.dragon_raja \
      --run-to-key-frame --save-screenshots --verbose \
      --render-test-output-dir=/tmp/g

When the second trace is run independently, the resulting pixels are
slightly different.

This CL disables batching tests on Android so each trace runs as a new
instrumentation/activity. This way the previous trace orientation does
not seem to matter.

The impact on shard runtime seems minor.

Bug: b/367401313
Change-Id: If1c41e45d063d434f92d46be8dd381e645a3ff0a
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5866850
Reviewed-by: Cody Northrop <[email protected]>
Reviewed-by: Yuly Novikov <[email protected]>
Commit-Queue: Roman Lavrov <[email protected]>
  • Loading branch information
romanl-g authored and Angle LUCI CQ committed Sep 17, 2024
1 parent 89f9ea0 commit 0f4c807
Showing 1 changed file with 10 additions and 2 deletions.
12 changes: 10 additions & 2 deletions src/tests/restricted_traces/restricted_trace_gold_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,14 @@ def _run_tests(args, tests, extra_flags, env, screenshot_dir, results, test_resu
if args.isolated_script_test_filter:
traces = angle_test_util.FilterTests(traces, args.isolated_script_test_filter)

batches = _get_batches(traces, args.batch_size)
if angle_test_util.IsAndroid():
# On Android, screen orientation changes between traces can result in small pixel diffs
# making results depend on the ordering of traces. Disable batching.
batch_size = 1
else:
batch_size = args.batch_size

batches = _get_batches(traces, batch_size)

for batch in batches:
if angle_test_util.IsAndroid():
Expand Down Expand Up @@ -454,7 +461,8 @@ def main():
default=0)
parser.add_argument(
'--batch-size',
help='Number of tests to run in a group. Default: %d' % DEFAULT_BATCH_SIZE,
help='Number of tests to run in a group. Default: %d (disabled on Android)' %
DEFAULT_BATCH_SIZE,
type=int,
default=DEFAULT_BATCH_SIZE)
parser.add_argument(
Expand Down

0 comments on commit 0f4c807

Please sign in to comment.