An issue by nolar at 2019-10-23 17:05:06+00:00
Original URL: zalando-incubator/kopf#212
 

Expected Behavior

The tests with timing use the explicit hard-coded time shifts, and do not fail based of luck or lack of luck.

Actual Behavior

The tests with timing fail randomly since the execution duration varies on a case by case basis.

Example:

• https://travis-ci.org/zalando-incubator/kopf/jobs/593947272#L1428

Notes

The tests with timings currently use minimised sleep/wait durations and timeouts — to make the whole test-run fast enough. However, this sometimes reaches the measurement error (the code itself takes little but noticeable time).

As a good solution, the sleeps/timeouts should be increased to values far above the code execution durations, and the tests should explicitly shift the time on demand beyond those increased sleeps/timeouts.

There is a tool called freezegun to time-travel during the tests. It does not cover the asyncio sleeps/waits, however. Here is an issue for freezegun: spulec/freezegun#290

Here is a proposed hack to mock asyncio.sleep() and loop.time() methods: https://stackoverflow.com/a/57748106/857383.

Any hack suitable for Kopf-specific testing (not necessary a generic all-purpose solution) would be fine.