gh-103092: Isolate _decimal

[CharlieZhao95]

This PR is based on @erlend-aasland 's great work. Currently, most of the test cases can be passed, but still some work needs to be done. I make this PR as a draft so anyone can remind me of missing work or improve this code.

TODO

• Test references leak and performance
• Solve compilation warnings
• Convert cond_map to heap type (not recorded in global-to-fix, but it seems like we should handle it)
• Adapt multi-phase initialization
• Remove redundant code and improve some dirty code

• Issue: Isolate Stdlib Extension Modules #103092

[erlend-aasland]

• Convert cond_map to heap type (not recorded in global-to-fix, but it seems like we should handle it)

I'm not sure I follow; cond_map is not a type.

Remove redundant code and improve some dirty code

Can you please spell this out? We cannot guess what you mean by this :)

[CharlieZhao95]

• Convert cond_map to heap type (not recorded in global-to-fix, but it seems like we should handle it)

I'm not sure I follow; cond_map is not a type.

Background

When I executed test_decimal.py after moving all variables in global-to-fix.csv into decimal_state, it raised an error:

Traceback (most recent call last):
  File "F:\CPython\github\cpython\Lib\test\test_decimal.py", line 5900, in <module>
    test_main(arith=True, verbose=True)
  File "F:\CPython\github\cpython\Lib\test\test_decimal.py", line 5840, in test_main
    init(C)
  File "F:\CPython\github\cpython\Lib\test\test_decimal.py", line 110, in init
    DefaultTestContext = m.Context(
                         ^^^^^^^^^^
KeyError: 'invalid signal dict'

I found that the place where the error occurs is PyDict_GetItemWithError(val, cm->ex) in dict_as_flags. This seems to be caused by the static signal_map whose members are assigned to types in module state.

So I guess signal_map should be moved to decimal_state (you can view my latest commit, maybe it's not the best solution). Likewise, cond_map[i].ex will be assigned to signal_map[i].ex, so it probably needs to be moved as well.

I'm not sure if these works are necessary, please correct me if I'm wrong :)

Remove redundant code and improve some dirty code

Can you please spell this out? We cannot guess what you mean by this :)

Of course, there is currently some unclean code. For example, signal_map_init or some code with FIXME comments, they work but are ugly. I will list the code that need to be modified after I have finished thinking. :)

[erlend-aasland]

Remove redundant code and improve some dirty code

Can you please spell this out? We cannot guess what you mean by this :)

Of course, there is currently some unclean code. For example, signal_map_init or some code with FIXME comments, they work but are ugly. I will list the code that need to be modified after I have finished thinking. :)

Is this code added by this PR, or code that is already there? For the latter, please do not include such changes in this PR. Instead, create an issue (unless there already is one), explain what you intend to do, and create a separate PR. This PR should focus on isolating _decimal, and nothing else.

[CharlieZhao95]

If this code added by this PR, or code that is already there?

Sorry for not being clear. I mean the code added by this PR. I like to leave "FIXME" in my code to remind myself of potential work :)

[CharlieZhao95]

I tested the reference counts using erlend's script and they look well.

$./python measure.py
before=90491, after=93999
before=94000, after=94000
before=94000, after=94000
before=94000, after=94000
before=94000, after=94000

Also, I compared the execution time of the test_decimal between the current branch and python3.12, and there is no significant change in them. More detailed benchmark results can be viewed in Python Speed ​​Center after the merge.

[CharlieZhao95]

Of course, there is currently some unclean code. For example, signal_map_init or some code with FIXME comments, they work but are ugly. I will list the code that need to be modified after I have finished thinking. :)

The code that I can think of that may need to be improved is as follows:

1. If we call the mpd_setminalloc function multiple times, this function is called in decimal_exec, it will raise a warning. I temporarily use a static variable to ensure that it is not called multiple times. This doesn't seem pretty, any suggestions for improvement?
2. Should we move signal_map into decimal_state(to fix the failing test cases, please see comment), is there a better solution?

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit 58f0049 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

[erlend-aasland]

I'm sorry, but I don't have the bandwidth to review this in the near future.

[kumaraditya303]

guard with Py_DEBUG

[erlend-aasland]

IMO, we should break this up into multiple PRs, like we did with _io; it will be easier to bisect if (when) bugs appear. See also PEP-687 for suggestions on how to break this up.

[CharlieZhao95]

IMO, we should break this up into multiple PRs, like we did with _io; it will be easier to bisect if (when) bugs appear. See also PEP-687 for suggestions on how to break this up.

Thanks. This PR made a lot of changes at the same time, which might make it difficult to review. But the _deciaml module is simpler than the _io module, and I think 4~5 PRs are enough.

I plan to break the PR into the following parts:

1. Establish a global module state and port some simple static types
2. Move other global static variables to the global module state
3. Convert the global module state to true module state

Potential work: Add Argument Clinic (It might be worth opening a new issue)

And I noticed that @kumaraditya303 has already done some review work, what do you think?

[kumaraditya303]

SGTM, ping me when you are done,