Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement lazy loading of pygame submodules (surfarray, sndarray) #3232

Closed
wants to merge 6 commits into from
Closed

Implement lazy loading of pygame submodules (surfarray, sndarray) #3232

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
efa96e8
Implement lazy loading of surfarray, sndarray submodules
aatle Nov 21, 2024
146cf90
Fix bug in lazy loading
aatle Nov 21, 2024
7fcd8e5
Shush pylint
aatle Nov 21, 2024
df1f23d
Add explicit imports for lazily loaded modules
aatle Nov 25, 2024
e22b31f
Improve documentation of new lazy loading system
aatle Nov 25, 2024
e72bd1e
Add reasoning for lazy import change in module docs
aatle Nov 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/reST/ref/sndarray.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ Each sample is an 8-bit or 16-bit integer, depending on the data format. A
stereo sound file has two values per sample, while a mono sound file only has
one.

.. versionchanged:: 2.5.3 sndarray module is lazily loaded to avoid loading NumPy needlessly

.. function:: array

| :sl:`copy Sound samples into an array`
Expand Down
2 changes: 2 additions & 0 deletions docs/reST/ref/surfarray.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ pixels from the surface and any changes performed to the array will make changes
in the surface. As this last functions share memory with the surface, this one
will be locked during the lifetime of the array.

.. versionchanged:: 2.5.3 surfarray module is lazily loaded to avoid loading NumPy needlessly

.. function:: array2d

| :sl:`Copy pixels into a 2d array`
Expand Down
50 changes: 40 additions & 10 deletions src_py/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ class MissingModule:
_NOT_IMPLEMENTED_ = True

def __init__(self, name, urgent=0):
import sys # pylint: disable=reimported

self.name = name
exc_type, exc_msg = sys.exc_info()[:2]
self.info = str(exc_msg)
Expand Down Expand Up @@ -259,7 +261,41 @@ def PixelArray(surface): # pylint: disable=unused-argument
except (ImportError, OSError):
transform = MissingModule("transform", urgent=1)


# lastly, the "optional" pygame modules

# Private, persisting alias for use in __getattr__
_MissingModule = MissingModule
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would make sense to store this as a local variable on __getattr__

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The MissingModule class is deleted at the end of the module, and __getattr__ still needs it, so it must be stored somewhere (not as local variable).
A few options:

  • Private alias of MissingModule
  • Move MissingModule to another file
  • Bind MissingModule as a default keyword argument in __getattr__

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an idea:

>>> a = 36
>>>
>>> def func():
...     print(func.b)
...
>>>
>>> func.b = a
>>> del a
>>> func()
36

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well I suggested that before but ultimately opted for a private alias because function attributes are kind of weird and dynamic. From my reasoning, it would've look strange and I would rather avoid these rare python features.
If it is necessary to avoid exposing _MissingModule in the pygame namespace, then sure, but I think the current approach is okay.



def __getattr__(name):
Starbuck5 marked this conversation as resolved.
Show resolved Hide resolved
"""Implementation of lazy loading for some optional pygame modules.

The surfarray and sndarray submodules use numpy, so they are loaded
lazily to avoid a heavy numpy import if the modules are never used.

The first access of a lazily loaded submodule loads it and sets it
as an attribute on the pygame module. Pygame itself doesn't import these modules.
If the first access is an attribute access and not an import, then __getattr__ is
invoked (as the attribute isn't set yet), which imports the module dynamically.

All lazy submodules are directly referenced in the packager_imports function.
"""
from importlib import import_module

LAZY_MODULES = "surfarray", "sndarray"
if name not in LAZY_MODULES:
# Normal behavior for attribute accesses that aren't lazy modules
raise AttributeError(f"module '{__name__}' has no attribute '{name}'")
try:
module = import_module(f"{__name__}.{name}")
# A successful import automatically sets the module attribute on the package
except (ImportError, OSError):
module = _MissingModule(name, urgent=0)
globals()[name] = module
return module


if "PYGAME_FREETYPE" in os.environ:
try:
import pygame.ftfont as font
Expand Down Expand Up @@ -300,16 +336,6 @@ def PixelArray(surface): # pylint: disable=unused-argument
except (ImportError, OSError):
scrap = MissingModule("scrap", urgent=0)

try:
import pygame.surfarray
except (ImportError, OSError):
surfarray = MissingModule("surfarray", urgent=0)

try:
import pygame.sndarray
except (ImportError, OSError):
sndarray = MissingModule("sndarray", urgent=0)

try:
import pygame._debug
from pygame._debug import print_debug_info
Expand Down Expand Up @@ -366,6 +392,10 @@ def packager_imports():
import pygame.macosx
import pygame.colordict

# lazily loaded pygame modules, just in case
import pygame.surfarray
import pygame.sndarray


# make Rects pickleable

Expand Down
Loading