Excessive CPU and RAM usage #29

yochananmarqos · 2023-12-24T21:15:47Z

This is a fine GUI, however as the title says, it's very intensive. I'm using 1.2.1 via my speedtest-librespeed AUR package on Manjaro GNOME (unstable branch).

The speedtest succeeds, however the terminal output seems to complain while it runs:

❯ speedtest-librespeed
Trying to fetch servers...
Exception ignored in: <coroutine object LibrespeedBackend.get_servers.<locals>.check_server at 0x7f5ae4d78840>
Traceback (most recent call last):
  File "/usr/share/speedtest/speedtest/backends/librespeed.py", line 37, in check_server
    async with session.get(server.pingURL, timeout=aiohttp.ClientTimeout(total=2.0)) as _:
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 574, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 544, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 911, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 1204, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 992, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1112, in create_connection
    transport, protocol = await self._create_connection_transport(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1147, in _create_connection_transport
    transport.close()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 112, in close
    self._ssl_protocol._start_shutdown()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 617, in _start_shutdown
    self._abort()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 667, in _abort
    self._transport.abort()
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 821, in abort
    self._force_close(None)
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 891, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object LibrespeedBackend.get_servers.<locals>.check_server at 0x7f5ae4d78940>
Traceback (most recent call last):
  File "/usr/share/speedtest/speedtest/backends/librespeed.py", line 37, in check_server
    async with session.get(server.pingURL, timeout=aiohttp.ClientTimeout(total=2.0)) as _:
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 574, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 544, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 911, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 1204, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 992, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1112, in create_connection
    transport, protocol = await self._create_connection_transport(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1147, in _create_connection_transport
    transport.close()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 112, in close
    self._ssl_protocol._start_shutdown()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 617, in _start_shutdown
    self._abort()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 667, in _abort
    self._transport.abort()
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 821, in abort
    self._force_close(None)
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 891, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object LibrespeedBackend.get_servers.<locals>.check_server at 0x7f5ae4d79440>
Traceback (most recent call last):
  File "/usr/share/speedtest/speedtest/backends/librespeed.py", line 37, in check_server
    async with session.get(server.pingURL, timeout=aiohttp.ClientTimeout(total=2.0)) as _:
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 574, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 544, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 911, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 1204, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 992, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1112, in create_connection
    transport, protocol = await self._create_connection_transport(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1147, in _create_connection_transport
    transport.close()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 112, in close
    self._ssl_protocol._start_shutdown()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 617, in _start_shutdown
    self._abort()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 667, in _abort
    self._transport.abort()
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 821, in abort
    self._force_close(None)
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 891, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object LibrespeedBackend.get_servers.<locals>.check_server at 0x7f5ae4d79d40>
Traceback (most recent call last):
  File "/usr/share/speedtest/speedtest/backends/librespeed.py", line 37, in check_server
    async with session.get(server.pingURL, timeout=aiohttp.ClientTimeout(total=2.0)) as _:
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 574, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 544, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 911, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 1204, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 992, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1112, in create_connection
    transport, protocol = await self._create_connection_transport(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1147, in _create_connection_transport
    transport.close()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 112, in close
    self._ssl_protocol._start_shutdown()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 617, in _start_shutdown
    self._abort()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 667, in _abort
    self._transport.abort()
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 821, in abort
    self._force_close(None)
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 891, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Task was destroyed but it is pending!
task: <Task pending name='Task-62' coro=<TCPConnector._resolve_host() running at /usr/lib/python3.11/site-packages/aiohttp/connector.py:884> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/lib/python3.11/asyncio/futures.py:387, Task.task_wakeup()]> cb=[shield.<locals>._inner_done_callback() at /usr/lib/python3.11/asyncio/tasks.py:891]>

The text was updated successfully, but these errors were encountered:

nekohayo · 2024-06-26T18:46:37Z

The tracebacks do not happen during operation, but after you close the app with the "X" button, as far as I can tell.

My total wild-guess hunch is that probably your GUI is spamming the graphics stack by refreshing way too often (the dev could probably throttle the labels + charts updates? If they update every milisecond or so, that would be very wasteful), and/or spamming glib signals, or something like that. I haven't looked at any code, this is just a guess.

More objectively, the developer can confirm whether that hunch of mine is true or not based on what we can see from profiling with Sysprof 46 (devs, let me know if you need the sysprof capture file), as shown below.

Flame graph of most frequent/expensive function calls:

Graphics marks (solid blocks like that seem to lead credence to my hypothesis that you are spamming the graphics stack):

yochananmarqos · 2024-06-26T18:56:05Z

Ah, you're right about the tracebacks happening after closing the application. Sorry, I didn't realize that's when it actually occurred.

Everything else you said is way over my head. I'm not "doing" anything other than running the program.

nekohayo · 2024-07-03T14:36:26Z

With "you" I meant the developer generally.

Another quick hunch on my theory above: whatever is happening, it's causing about 62 thousand function calls in the app over the span of 29.6 seconds. That's a bit over 2000 function calls per second, which seems very wasteful, I don't see why you'd need more than, say, 60 calls per second on a 60 Hz monitor based on vsync. Even on high refresh rate monitors at 240hz, it would still be ten times less than what is happening now.

nekohayo · 2024-07-03T15:42:23Z

Just to be sure, as I narrowed down the tracebacks to occurring only outside of the flatpak version and causing incorrect measurements (see issue #37), I have retested the performance issue here with the flatpak version, and it still happens; 100% single-core CPU usage, so the function calls spammage performance issue itself is apparently not caused by the tracebacks (and it happens even before the upload test occurs anyway).

nekohayo mentioned this issue Jul 3, 2024

With aiohttp 3.9.x and newer, frequent Python tracebacks during benchmark cause upload speed measurement to be incorrect #37

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Excessive CPU and RAM usage #29

Excessive CPU and RAM usage #29

yochananmarqos commented Dec 24, 2023 •

edited

Loading

nekohayo commented Jun 26, 2024 •

edited

Loading

yochananmarqos commented Jun 26, 2024

nekohayo commented Jul 3, 2024

nekohayo commented Jul 3, 2024

Excessive CPU and RAM usage #29

Excessive CPU and RAM usage #29

Comments

yochananmarqos commented Dec 24, 2023 • edited Loading

nekohayo commented Jun 26, 2024 • edited Loading

yochananmarqos commented Jun 26, 2024

nekohayo commented Jul 3, 2024

nekohayo commented Jul 3, 2024

yochananmarqos commented Dec 24, 2023 •

edited

Loading

nekohayo commented Jun 26, 2024 •

edited

Loading