[meta issue] PEP 3121, 384 Refactoring

[abalkin]

BPO	15787
Nosy	@loewis, @ncoghlan, @abalkin, @pitrou, @vstinner
Dependencies	bpo-14732: PEP 3121 Refactoring applied to _csv module bpo-14935: PEP 384 Refactoring applied to _csv module bpo-14936: PEP 3121, 384 refactoring applied to curses_panel module bpo-15140: PEP 384 inconsistent with implementation bpo-15389: PEP 3121, 384 refactoring applied to curses module bpo-15390: PEP 3121, 384 refactoring applied to _datetime module bpo-15650: PEP 3121, 384 refactoring applied to dbm module bpo-15651: PEP 3121, 384 refactoring applied to elementtree module bpo-15652: PEP 3121, 384 refactoring applied to gdbm module bpo-15653: PEP 3121, 384 refactoring applied to hashopenssl module bpo-15666: PEP 3121, 384 refactoring applied to lzma module bpo-15667: PEP 3121, 384 refactoring applied to pickle module bpo-15668: PEP 3121, 384 Refactoring applied to random module bpo-15669: PEP 3121, 384 Refactoring applied to sre module bpo-15670: PEP 3121, 384 Refactoring applied to ssl module bpo-15671: PEP 3121, 384 Refactoring applied to struct module bpo-15672: PEP 3121, 384 Refactoring applied to testbuffer module bpo-15673: PEP 3121, 384 Refactoring applied to testcapi module bpo-15674: PEP 3121, 384 Refactoring applied to _thread module bpo-15675: PEP 3121, 384 Refactoring applied to array module bpo-15680: PEP 3121 refactoring applied to audioop module bpo-15681: PEP 3121 refactoring applied to binascii module bpo-15682: PEP 3121 refactoring applied to fpectl module bpo-15684: PEP 3121 refactoring applied to fpetest module bpo-15685: PEP 3121, 384 Refactoring applied to itertools module bpo-15686: PEP 3121, 384 Refactoring applied to md5 module bpo-15687: PEP 3121, 384 Refactoring applied to mmap module bpo-15688: PEP 3121 Refactoring applied to nis module bpo-15689: PEP 3121, 384 Refactoring applied to operator module bpo-15690: PEP 3121, 384 Refactoring applied to parser module bpo-15691: PEP 3121, 384 Refactoring applied to posix module bpo-15697: PEP 3121 refactoring applied to pwd module bpo-15698: PEP 3121, 384 Refactoring applied to pyexpat module bpo-15700: PEP 3121, 384 Refactoring applied to resource module bpo-15703: PEP 3121, 384 Refactoring applied to select module bpo-15704: PEP 3121, 384 Refactoring applied to sha1 module bpo-15705: PEP 3121, 384 Refactoring applied to sha256 module bpo-15706: PEP 3121, 384 Refactoring applied to sha512 module bpo-15707: PEP 3121, 384 Refactoring applied to signal module bpo-15708: PEP 3121, 384 Refactoring applied to socket module bpo-15709: PEP 3121, 384 Refactoring applied to termios module bpo-15711: PEP 3121, 384 Refactoring applied to time module bpo-15712: PEP 3121, 384 Refactoring applied to unicodedata module bpo-15713: PEP 3121, 384 Refactoring applied to zipimport module bpo-15714: PEP 3121, 384 Refactoring applied to grp module bpo-15721: PEP 3121, 384 Refactoring applied to tkinter module bpo-15722: PEP 3121, 384 Refactoring applied to decimal module bpo-15733: PEP 3121, 384 Refactoring applied to winapi module bpo-15734: PEP 3121, 384 Refactoring applied to spwd module bpo-15735: PEP 3121, 384 Refactoring applied to ossaudio module bpo-15848: PEP 3121, 384 Refactoring applied to xxsubtype module bpo-15849: PEP 3121, 384 Refactoring applied to xx module bpo-15884: PEP 3121, 384 Refactoring applied to ctypes module

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2012-08-27.03:36:04.806>
labels = ['extension-modules', 'type-feature', '3.11']
title = '[meta issue] PEP 3121, 384 Refactoring'
updated_at = <Date 2021-07-28.22:21:25.308>
user = 'https://github.com/abalkin'

bugs.python.org fields:

activity = <Date 2021-07-28.22:21:25.308>
actor = 'belopolsky'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2012-08-27.03:36:04.806>
creator = 'belopolsky'
dependencies = ['14732', '14935', '14936', '15140', '15389', '15390', '15650', '15651', '15652', '15653', '15666', '15667', '15668', '15669', '15670', '15671', '15672', '15673', '15674', '15675', '15680', '15681', '15682', '15684', '15685', '15686', '15687', '15688', '15689', '15690', '15691', '15697', '15698', '15700', '15703', '15704', '15705', '15706', '15707', '15708', '15709', '15711', '15712', '15713', '15714', '15721', '15722', '15733', '15734', '15735', '15848', '15849', '15884']
files = []
hgrepos = []
issue_num = 15787
keywords = []
message_count = 8.0
messages = ['169177', '169179', '194600', '194602', '194889', '195098', '381428', '398434']
nosy_count = 6.0
nosy_names = ['loewis', 'ncoghlan', 'belopolsky', 'pitrou', 'vstinner', 'Robin.Schreiber']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue15787'
versions = ['Python 3.11']

[abalkin]

This is a meta-issue to keep track of a global PEP-3121 refactoring effort. Per-module issues will be added as dependencies. Let's move the discussion that is not specific to any particular module here.

[abalkin]

Robin,

Perhaps we should start with the "xx" modules: Modules/xxmodule.c and Modules/xxsubtype.c. These modules server as a demonstration of best practices and it is natural to use them to iron out any refactoring issues without any risk to deployed code.

[abalkin]

"""
Regarding the suggestion of separating PEP-3121 and PEP-384. It might be true that datetime and other modules do not benefit directly from PEP-384, however it is still a fact that the stdlib modules should be seen as a set of reference modules, that are all implemented in a way that complies with the implementation fo the xxmodules.
I have talked with Martin von Löwis about this, and as far as I understood him correctly he also sees the PEP-384 refactoring applied to the whole stdlib as a necessary "signal" to other developers to refactor their modules accordingly.
""" (Robin Schreiber, bpo-15390, msg177274)

MvL have recently confirmed this on python-dev: "Choice of supporting PEP-384 was deliberate. It will change all types into heap types, which is useful for multiple-interpreter support and GC."

Accordingly, I've changed the title of this issue and added a few PEP-384 only dependencies.

[abalkin]

With respect to PEP-384 refactoring, I would like to see Tools/scripts/abitype.py used for most of the conversions. The PEP itself can probably be amended to advertise this tool more prominently.

[RobinSchreiber]

I have in fact used abitype.py to produce all of my PEP-384 patches, however it failed to work correctly in like 50% of all cases, and I had to complete the rest of the patch by hand.I thought about correcting the abitype.py throughout the GSOC, but I happened to find it easier to simply do the missing steps by hand. (I don't know If the script has been fixed up to this point though).
In any case, it might also be interesting for external extension module developers to make use of a fully working version of this script, so they can make their modules PEP-384 compliant without investing too much time.
One has to keep in mind however that almost any script will fail to work by itself, once we run into problems like refcounting or concurrency when applying the patch to a module.

I will have some free time throughout next week and will start to correct the patches for the xx modules (as Alex proposed), and continue from there to all the other patches I have submitted a year ago.

I am deeply sorry that I have not continued my work on this project earlier, however I dramatically underestimated the amount if work related to university, etc... I still feel obliged to complete all these patches.

[abalkin]

I strongly believe that it is worthwhile to invest in fixing abitype.py. It is much easier to review a patch to one python script than to review 50+ patches to C files. There is no excuse for this tool not to work on all stdlib modules. If there are any specific issues with the way individual modules are written that prevent automatic conversion, I would prefer to make a coding style change first and then include all modules in one automated PEP-384 conversion.

[vstinner]

The work is now tracked at bpo-1635741.

[abalkin]

The work is now tracked at bpo-1635741.

Which work? bpo-1635741 does not appear to be a meta-issue and in msg381432 it loops back here.

[AA-Turner]

The linked #44470 has been closed, closing this one.

A