Skip to content

Commit

Permalink
pythongh-102471: Add PyLong import and export API
Browse files Browse the repository at this point in the history
Co-authored-by: Sergey B Kirpichev <[email protected]>
  • Loading branch information
vstinner and skirpichev committed Aug 6, 2024
1 parent 4c31791 commit f4fdbf2
Show file tree
Hide file tree
Showing 10 changed files with 512 additions and 3 deletions.
153 changes: 153 additions & 0 deletions Doc/c-api/long.rst
Original file line number Diff line number Diff line change
Expand Up @@ -540,10 +540,163 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
Exactly what values are considered compact is an implementation detail
and is subject to change.
.. versionadded:: 3.12
.. c:function:: Py_ssize_t PyUnstable_Long_CompactValue(const PyLongObject* op)
If *op* is compact, as determined by :c:func:`PyUnstable_Long_IsCompact`,
return its value.
Otherwise, the return value is undefined.
.. versionadded:: 3.12
Export API
^^^^^^^^^^
.. versionadded:: 3.14
.. c:type:: Py_digit
A single unsigned digit in the range [``0``; ``PyLong_BASE - 1``].
It is usually used in an *array of digits*, such as the
:c:member:`PyLong_DigitArray.digits` array.
Its size depend on the :c:macro:`!PYLONG_BITS_IN_DIGIT` macro:
see the ``configure`` :option:`--enable-big-digits` option.
See :c:member:`PyLong_LAYOUT.bits_per_digit` for the number of bits per
digit and :c:member:`PyLong_LAYOUT.digit_size` for the size of a digit (in
bytes).
.. c:struct:: PyLong_LAYOUT
Layout of an array of digits, used by Python :class:`int` object.
See also :attr:`sys.int_info` which exposes similar information to Python.
.. c:member:: uint8_t bits_per_digit
Bits per digit.
.. c:member:: uint8_t digit_size
Digit size in bytes.
.. c:member:: int8_t word_endian
Word endian:
- ``1`` for most significant word first (big endian)
- ``-1`` for least significant first (little endian)
.. c:member:: int8_t array_endian
Array endian:
- ``1`` for most significant byte first (big endian)
- ``-1`` for least significant first (little endian)
.. c:struct:: PyLong_DigitArray
A Python :class:`int` object exported as an array of digits.
See :c:struct:`PyLong_LAYOUT` for the :c:member:`digits` layout.
.. c:member:: PyObject *obj
Strong reference to the Python :class:`int` object.
.. c:member:: int negative
1 if the number is negative, 0 otherwise.
.. c:member:: Py_ssize_t ndigits
Number of digits in :c:member:`digits` array.
.. c:member:: const Py_digit *digits
Read-only array of unsigned digits.
.. c:function:: int PyLong_AsDigitArray(PyObject *obj, PyLong_DigitArray *array)
Export a Python :class:`int` object as an array of digits.
On success, set *\*array* and return 0.
On error, set an exception and return -1.
This function always succeeds if *obj* is a Python :class:`int` object or a
subclass.
:c:func:`PyLong_FreeDigitArray` must be called once done with using
*export*.
.. c:function:: void PyLong_FreeDigitArray(PyLong_DigitArray *array)
Release the export *array* created by :c:func:`PyLong_AsDigitArray`.
PyLongWriter API
^^^^^^^^^^^^^^^^
The :c:type:`PyLongWriter` API can be used to import an integer.
.. versionadded:: 3.14
.. c:struct:: PyLongWriter
A Python :class:`int` writer instance.
The instance must be destroyed by :c:func:`PyLongWriter_Finish`.
.. c:function:: PyLongWriter* PyLongWriter_Create(int negative, Py_ssize_t ndigits, Py_digit **digits)
Create a :c:type:`PyLongWriter`.
On success, set *\*digits* and return a writer.
On error, set an exception and return ``NULL``.
*negative* is ``1`` if the number is negative, or ``0`` otherwise.
*ndigits* is the number of digits in the *digits* array. It must be
positive.
The caller must initialize the array of digits *digits* and then call
:c:func:`PyLongWriter_Finish` to get a Python :class:`int`. Digits must be
in the range [``0``; ``PyLong_BASE - 1``]. Unused digits must be set to
``0``.
See :c:struct:`PyLong_LAYOUT` for the layout of an array of digits.
.. c:function:: PyObject* PyLongWriter_Finish(PyLongWriter *writer)
Finish a :c:type:`PyLongWriter` created by :c:func:`PyLongWriter_Create`.
On success, return a Python :class:`int` object.
On error, set an exception and return ``NULL``.
Example creating an integer from an array of digits::
PyObject *
long_import(int negative, Py_ssize_t ndigits, Py_digit *digits)
{
Py_digit *writer_digits;
PyLongWriter *writer = PyLongWriter_Create(negative, ndigits,
&writer_digits);
if (writer == NULL) {
return NULL;
}
memcpy(writer_digits, digits, ndigits * sizeof(digit));
return PyLongWriter_Finish(writer);
}
2 changes: 2 additions & 0 deletions Doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,8 @@
('c:type', 'size_t'),
('c:type', 'ssize_t'),
('c:type', 'time_t'),
('c:type', 'int8_t'),
('c:type', 'uint8_t'),
('c:type', 'uint64_t'),
('c:type', 'uintmax_t'),
('c:type', 'uintptr_t'),
Expand Down
3 changes: 2 additions & 1 deletion Doc/using/configure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,8 @@ General Options

Define the ``PYLONG_BITS_IN_DIGIT`` to ``15`` or ``30``.

See :data:`sys.int_info.bits_per_digit <sys.int_info>`.
See :data:`sys.int_info.bits_per_digit <sys.int_info>` and the
:c:type:`Py_digit` type.

.. option:: --with-suffix=SUFFIX

Expand Down
10 changes: 10 additions & 0 deletions Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
Expand Up @@ -405,6 +405,16 @@ New Features

(Contributed by Victor Stinner in :gh:`119182`.)

* Add a new import and export API for Python :class:`int` objects:

* :c:func:`PyLong_AsDigitArray`;
* :c:func:`PyLong_FreeDigitArray`;
* :c:func:`PyLongWriter_Create`;
* :c:func:`PyLongWriter_Finish`;
* :c:struct:`PyLong_LAYOUT`.

(Contributed by Victor Stinner in :gh:`102471`.)

Porting to Python 3.14
----------------------

Expand Down
52 changes: 50 additions & 2 deletions Include/cpython/longintrepr.h
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,10 @@ typedef long stwodigits; /* signed variant of twodigits */
#else
#error "PYLONG_BITS_IN_DIGIT should be 15 or 30"
#endif
#define PyLong_BASE ((digit)1 << PyLong_SHIFT)
#define PyLong_MASK ((digit)(PyLong_BASE - 1))
#define PyLong_BASE ((Py_digit)1 << PyLong_SHIFT)
#define PyLong_MASK ((Py_digit)(PyLong_BASE - 1))

typedef digit Py_digit;

/* Long integer representation.
Expand Down Expand Up @@ -139,6 +141,52 @@ _PyLong_CompactValue(const PyLongObject *op)
#define PyUnstable_Long_CompactValue _PyLong_CompactValue


/* --- Import/Export API -------------------------------------------------- */

typedef struct PyLongLayout {
// Bits per digit
uint8_t bits_per_digit;

// Digit size in bytes
uint8_t digit_size;

// Word endian:
// * 1 for most significant word first (big endian)
// * -1 for least significant first (little endian)
int8_t word_endian;

// Array endian:
// * 1 for most significant byte first (big endian)
// * -1 for least significant first (little endian)
int8_t array_endian;
} PyLongLayout;

PyAPI_DATA(const PyLongLayout) PyLong_LAYOUT;

typedef struct PyLong_DigitArray {
PyObject *obj;
int negative;
Py_ssize_t ndigits;
const Py_digit *digits;
} PyLong_DigitArray;

PyAPI_FUNC(int) PyLong_AsDigitArray(
PyObject *obj,
PyLong_DigitArray *array);
PyAPI_FUNC(void) PyLong_FreeDigitArray(
PyLong_DigitArray *array);


/* --- PyLongWriter API --------------------------------------------------- */

typedef struct PyLongWriter PyLongWriter;

PyAPI_FUNC(PyLongWriter*) PyLongWriter_Create(
int negative,
Py_ssize_t ndigits,
Py_digit **digits);
PyAPI_FUNC(PyObject*) PyLongWriter_Finish(PyLongWriter *writer);

#ifdef __cplusplus
}
#endif
Expand Down
66 changes: 66 additions & 0 deletions Lib/test/test_capi/test_long.py
Original file line number Diff line number Diff line change
Expand Up @@ -631,6 +631,72 @@ def test_long_getsign(self):

# CRASHES getsign(NULL)

def test_long_layout(self):
# Test PyLong_LAYOUT
int_info = sys.int_info
layout = _testcapi.get_pylong_layout()
expected = {
'array_endian': -1,
'bits_per_digit': int_info.bits_per_digit,
'digit_size': int_info.sizeof_digit,
'word_endian': -1 if sys.byteorder == 'little' else 1,
}
self.assertEqual(layout, expected)

def test_long_export(self):
# Test PyLong_Export()
layout = _testcapi.get_pylong_layout()
base = 2 ** layout['bits_per_digit']

pylong_export = _testcapi.pylong_export
self.assertEqual(pylong_export(0), (0, [0]))
self.assertEqual(pylong_export(123), (0, [123]))
self.assertEqual(pylong_export(-123), (1, [123]))
self.assertEqual(pylong_export(base**2 * 3 + base * 2 + 1),
(0, [1, 2, 3]))

with self.assertRaises(TypeError):
pylong_export(1.0)
with self.assertRaises(TypeError):
pylong_export(0+1j)
with self.assertRaises(TypeError):
pylong_export("abc")

def test_longwriter_create(self):
# Test PyLong_Import()
layout = _testcapi.get_pylong_layout()
base = 2 ** layout['bits_per_digit']

pylongwriter_create = _testcapi.pylongwriter_create
self.assertEqual(pylongwriter_create(0, []), 0)
self.assertEqual(pylongwriter_create(0, [0]), 0)
self.assertEqual(pylongwriter_create(0, [123]), 123)
self.assertEqual(pylongwriter_create(1, [123]), -123)
self.assertEqual(pylongwriter_create(1, [1, 2]),
-(base * 2 + 1))
self.assertEqual(pylongwriter_create(0, [1, 2, 3]),
base**2 * 3 + base * 2 + 1)
max_digit = base - 1
self.assertEqual(pylongwriter_create(0, [max_digit, max_digit, max_digit]),
base**2 * max_digit + base * max_digit + max_digit)

# normalize
self.assertEqual(pylongwriter_create(0, [123, 0, 0]), 123)

# test singletons + normalize
for num in (-2, 0, 1, 5, 42, 100):
self.assertIs(pylongwriter_create(bool(num < 0), [abs(num), 0]),
num)

# round trip: Python int -> export -> Python int
pylong_export = _testcapi.pylong_export
numbers = [*range(0, 10), 12345, 0xdeadbeef, 2**100, 2**100-1]
numbers.extend(-num for num in list(numbers))
for num in numbers:
with self.subTest(num=num):
export = pylong_export(num)
self.assertEqual(pylongwriter_create(*export), num, export)


if __name__ == "__main__":
unittest.main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Add a new import and export API for Python :class:`int` objects:

* :c:func:`PyLong_AsDigitArray`;
* :c:func:`PyLong_FreeDigitArray`;
* :c:func:`PyLongWriter_Create`;
* :c:func:`PyLongWriter_Finish`;
* :c:struct:`PyLong_LAYOUT`.

Patch by Victor Stinner.
Loading

0 comments on commit f4fdbf2

Please sign in to comment.