tests fail at midnight?

[tuukka]

Time is 00:23 and the tests fail in master (commit id 5c3e0d2) although they didn't earlier:

EE.E../home/tuukka/stop2.0-backend/venv/lib/python3.4/site-packages/werkzeug/local.py:343: DeprecationWarning: json is deprecated.  Use get_json() instead.
  return getattr(self._get_current_object(), name)
/home/tuukka/stop2.0-backend/venv/lib/python3.4/site-packages/paho/mqtt/publish.py:223: ResourceWarning: unclosed <socket.socket fd=12, family=AddressFamily.AF_INET, type=2049, proto=0, laddr=('127.0.0.1', 36712), raddr=('127.0.0.1', 39740)>
  multiple([msg], hostname, port, client_id, keepalive, will, auth, tls, protocol, transport)
/home/tuukka/stop2.0-backend/venv/lib/python3.4/site-packages/paho/mqtt/publish.py:223: ResourceWarning: unclosed <socket.socket fd=13, family=AddressFamily.AF_INET, type=2049, proto=0, laddr=('127.0.0.1', 39740)>
  multiple([msg], hostname, port, client_id, keepalive, will, auth, tls, protocol, transport)
/home/tuukka/stop2.0-backend/src/tests/test_stop.py:23: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(response.status_code, 200)
..
======================================================================
ERROR: test_get_busses_by_stop_id (test_services.TestDigitransitAPIService)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tuukka/stop2.0-backend/src/tests/test_services.py", line 42, in test_get_busses_by_stop_id
    stop = self.digitransitAPIService.get_busses_by_stop_id("HSL:1362141", 100)
  File "/home/tuukka/stop2.0-backend/src/services.py", line 81, in get_busses_by_stop_id
    data = json.loads(self.get_query(query))["data"]["stop"]
  File "/usr/lib/python3.4/json/__init__.py", line 318, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.4/json/decoder.py", line 343, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.4/json/decoder.py", line 361, in raw_decode
    raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 1 (char 0)

======================================================================
ERROR: test_get_stops (test_services.TestDigitransitAPIService)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tuukka/stop2.0-backend/src/tests/test_services.py", line 13, in test_get_stops
    stops = self.digitransitAPIService.get_stops(60.203978, 24.9633573, 160)
  File "/home/tuukka/stop2.0-backend/src/services.py", line 25, in get_stops
    stops.append({"stop": self.get_busses_by_stop_id(stop_id['stop_id'], stop_id['distance'])})
  File "/home/tuukka/stop2.0-backend/src/services.py", line 81, in get_busses_by_stop_id
    data = json.loads(self.get_query(query))["data"]["stop"]
  File "/usr/lib/python3.4/json/__init__.py", line 318, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.4/json/decoder.py", line 343, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.4/json/decoder.py", line 361, in raw_decode
    raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 1 (char 0)

======================================================================
ERROR: test_get_stops_near_coordinates (test_services.TestDigitransitAPIService)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tuukka/stop2.0-backend/src/tests/test_services.py", line 32, in test_get_stops_near_coordinates
    stoplist_returns_only_three = self.digitransitAPIService.get_stops_near_coordinates(60.203978, 24.9633573, 300)
  File "/home/tuukka/stop2.0-backend/src/services.py", line 46, in get_stops_near_coordinates
    data = data['data']['stopsByRadius']['edges']
KeyError: 'stopsByRadius'

----------------------------------------------------------------------
Ran 8 tests in 1.101s

FAILED (errors=3)
/usr/lib/python3.4/asyncio/base_events.py:359: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
/home/tuukka/stop2.0-backend/venv/lib/python3.4/importlib/_bootstrap.py:2150: ImportWarning: sys.meta_path is empty

[heileino]

This is now fixed by adding more night-time realtimeArrival values to the mock-hsl-api.

[heileino]

The testing failures seems to be still there when actually running tests after midnight. Couldn't get the same failures simulated at daytime. I will try to find out the reason for this.

[heileino]

Test errors disappeared at 2.00am EET. Seems to be some kind of datetime issue. After 2.00am test_services.py tests are ok again but test_stop.py has one failure:

Failure
Traceback (most recent call last):
File "/home/heikki/ohtu/stop2.0-backend/src/tests/test_stop.py", line 18, in test_unicode_destination_name
self.assertTrue(response.data.find(b"L\\u00e4nsiterminaali") != -1)
AssertionError: False is not true

This one should be solved quite easily by adding more proper nighttime data to the mock-hsl-api and I don't think it's related to the main issue here.

edit: Well, actually it looks like the "test_unicode_destination_name"-test fails because it's not using any mocked api data. So basically this test will work again at around 5am EET when the real life traffic starts. If the only function of this test is to test the functionality of unicode in response message, the easiest solution would be testing the suitable stop_name instead of the destination_name.

[tuukka]

Unfortunately, the unicode problem was not visible in stop_name but only in destination_name. Perhaps there could be a time parameter that could be used to test a fixed time of day?

[heileino]

The issue with ‘test_unicode_destination_name’ is now solved in branch ‘fixing_nighttime_issues’ (also merged to master now) by using freeze_time from FreezeGun library. Freeze_time could also be useful for some other time sensitive tests now and in the future.

The issue with test errors in original post seems to exists because a local hsl-mock-api docker container uses UTC time instead of the user’s local timezone time. I have successfully solved this either by running mock-hsl-api locally without a docker (e.g. in PyCharm) or by adding “-v /etc/localtime:/etc/localtime:ro” and “-v /etc/timezone:/etc/timezone:ro” to ‘docker run’ command in command line. There is also an option to add these two volumes to a Dockerfile or to a Docker Compose configure file but I have opted not to do that at this point because I don't have enough knowledge on possible issues with that. However, in my experience, the timezone issue is not there when pushing a branch to a remote branch with Travis CI. So my understanding is that the docker timezone issue is only potential problem in local testing environment.

My solution for adding volumes to a 'docker run' command is inspired on this: https://hub.docker.com/r/zinvoice/elasticsearch/.

There are also some other suggestions to solve this problem:
moby/moby#12084
https://www.ivankrizsan.se/2015/10/31/time-in-docker-containers/

[tuukka]

FreezeGun looks really useful for the tests! Regarding timezones, could stop2.0-backend always use UTC time? Or if you need the local timezone, could you always use the "Europe/Finland" timezone definition? This is what we do in the Digitransit backend for Java: https://github.com/HSLdevcom/OpenTripPlanner/blob/master/run.sh#L36

[tuukka]

(I have to add that any solution including yours is just fine at this point, though!)