diff --git a/peps/pep-0793.rst b/peps/pep-0793.rst index 868e14945e6..d0000fa76be 100644 --- a/peps/pep-0793.rst +++ b/peps/pep-0793.rst @@ -2,7 +2,7 @@ PEP: 793 Title: PyModExport: A new entry point for C extension modules Author: Petr Viktorin Discussions-To: https://discuss.python.org/t/93444 -Status: Accepted +Status: Final Type: Standards Track Created: 23-May-2025 Python-Version: 3.15 @@ -10,6 +10,8 @@ Post-History: `14-Mar-2025 `__, `27-May-2025 `__, Resolution: `23-Oct-2025 `__ +.. canonical-doc:: :ref:`py3.15:extension-modules` + Abstract ======== @@ -21,7 +23,7 @@ This allows extension authors to avoid using a statically allocated ``PyObject``, lifting the most common obstacle to making one compiled library file usable with both regular and free-threaded builds of CPython. -To make this viable, we also specify new module slot types to replace +To make this viable, we also specify new module slot IDs to replace ``PyModuleDef``'s fields, and to allow adding a *token* similar to the ``Py_tp_token`` used for type objects. @@ -131,8 +133,8 @@ This proposal does away with fixed fields and proposes using a slots array directly, without a wrapper struct. The ``PyModuleDef_Slot`` struct does have some downsides compared to fixed fields. -We believe these are fixable, but leave that out of scope of this PEP -(see “Improving slots in general” in the Possible Future Directions section). +We believe these are fixable, but leave that out of scope of this PEP. +(Note: this was done in :pep:`820`, still in Python 3.15.) Tokens @@ -187,6 +189,8 @@ like this: PyModuleDef_Slot *PyModExport_(void); +.. note:: :pep:`820` changed the return type to ``PySlot *``. + where ```` is the name of the module. For non-ASCII names, it will instead look for ``PyModExportU_``, with ```` encoded as for existing ``PyInitU_*`` hooks @@ -223,14 +227,13 @@ A new function will be added to create a module from an array of slots: PyObject *PyModule_FromSlotsAndSpec(const PyModuleDef_Slot *slots, PyObject *spec) +.. note:: :pep:`820` changed the first argument type to ``PySlot *``. + The *slots* argument must point to an array of ``PyModuleDef_Slot`` structures, terminated by a slot with ``slot=0`` (typically written as ``{0}`` in C). -There are no required slots, though *slots* must not be ``NULL``. -It follows that minimal input contains only the terminator slot. - -.. note:: - - If :pep:`803` is accepted, the ``Py_mod_abi`` slot will be mandatory. +The ``Py_mod_abi`` slot is required (see :pep:`803`); all other slots +are optional. +It follows that *slots* must not be ``NULL``. The *spec* argument is a duck-typed ModuleSpec-like object, meaning that any attributes defined for ``importlib.machinery.ModuleSpec`` have matching @@ -373,7 +376,7 @@ Bits & Pieces ------------- A ``PyMODEXPORT_FUNC`` macro will be added, similar to the ``PyMODINIT_FUNC`` -macro but with ``PyModuleDef_Slot *`` as the return type. +macro but with ``PySlot *`` as the return type. A ``PyModule_GetStateSize`` function will be added to retrieve the size set by ``Py_mod_state_size`` or ``PyModuleDef.m_size``. @@ -397,6 +400,9 @@ The ``PyInit_*`` export hook will be New API summary --------------- + +.. note:: This summary was updated with a change from :pep:`820`. + Python will load a new module export hook, with two variants: .. code-block:: c @@ -408,7 +414,7 @@ The following functions will be added: .. code-block:: c - PyObject *PyModule_FromSlotsAndSpec(const PyModuleDef_Slot *, PyObject *spec) + PyObject *PyModule_FromSlotsAndSpec(const PySlot *, PyObject *spec) int PyModule_Exec(PyObject *) int PyModule_GetToken(PyObject *, void**) PyObject *PyType_GetModuleByToken(PyTypeObject *type, const void *token) @@ -477,6 +483,15 @@ Here is a guide to convert an existing module to the new API, including some tricky edge cases. It should be moved to a HOWTO in the documentation. +.. note:: + + The guide is available at :ref:`py3.15:abi3t-howto-modexport`. + (It is part of the ``abi3t`` migration HOWTO since switching to + ``PyModExport`` doesn't bring benefits in 3.15 if you don't also + adopt ``abi3t``.) + + This section contains the original, outdated guide. + This guide is meant for hand-written modules. For code generators and language wrappers, the :ref:`pep793-shim` below may be more useful. @@ -585,9 +600,13 @@ The following implementation can be copied and pasted to a project; only the names ``PyInit_examplemodule`` (twice) and ``PyModExport_examplemodule`` should need adjusting. -When added to the :ref:`pep793-example` below and compiled with a -non-free-threaded build of this PEP's reference implementation, the resulting -extension is compatible with non-free-threading 3.9+ builds, in addition to a +.. note:: + + This section was updated for :pep:`820`. + +When compiled together with the :ref:`pep793-example` below on a +non-free-threaded build of Python 3.15, the resulting +extension is compatible with non-free-threading 3.11+ builds, in addition to a free-threading build of the reference implementation. (The module must be named without a version tag, e.g. ``examplemodule.so``, and be placed on ``sys.path``.) @@ -595,14 +614,15 @@ and be placed on ``sys.path``.) Full support for creating such modules will require backports of some new API, and support in build/install tools. This is out of scope of this PEP. (In particular, the demo “cheats” by using a subset of Limited API 3.15 that -*happens to work* on 3.9; a proper implementation would use Limited API 3.9 -with backport shims for new API like ``Py_mod_name``.) +*happens to work* on 3.11, and includes a few hacks. +A proper implementation would use Limited API 3.11 with cleaner backport shims +for new API like ``Py_mod_name``.) This implementation places a few additional requirements on the slots array: -- Slots that correspond to ``PyModuleDef`` members must come first. +- ``Py_mod_slots`` and ``Py_slot_subslots`` are not supported. - A ``Py_mod_name`` slot is required. -- Any ``Py_mod_token`` must be set to ``&module_def_and_token``, defined here. +- Any ``Py_mod_token`` must be set to the ``MOD_TOKEN`` defined here. .. literalinclude:: pep-0793/shim.c :language: c @@ -626,6 +646,10 @@ be added as a new HOWTO. Example ======= +.. note:: + + The example was updated for :pep:`820`. + .. literalinclude:: pep-0793/examplemodule.c :language: c @@ -693,6 +717,10 @@ These ideas are out of scope for *this* proposal. Improving slots in general -------------------------- +.. note:: + + This idea was implemented in :pep:`820`. + Slots -- and specifically the existing ``PyModuleDef_Slot`` -- do have a few shortcomings. The most important are: diff --git a/peps/pep-0793/examplemodule.c b/peps/pep-0793/examplemodule.c index c49fd0d628b..6063077c7e0 100644 --- a/peps/pep-0793/examplemodule.c +++ b/peps/pep-0793/examplemodule.c @@ -35,7 +35,13 @@ typedef struct { int value; } examplemodule_state; -static PyModuleDef_Slot examplemodule_slots[]; +static PySlot examplemodule_slots[]; + +#ifndef MOD_TOKEN +// Module token: normally set to the slots array, +// but a backwards-compatibility shim will redefine it. +#define MOD_TOKEN (&examplemodule_slots) +#endif // increment_value function @@ -59,20 +65,21 @@ exampletype_repr(PyObject *self) { /* To get module state, we cannot use PyModule_GetState(Py_TYPE(self)), * since Py_TYPE(self) might be a subclass defined in an unrelated module. - * So, use PyType_GetModuleByToken. + * So, we should use use PyType_GetModuleByToken. + * For pre-3.15 compatibility, we use PyType_GetModuleByDef instead: + * this needs a cast and returns a borrowed reference. */ - PyObject *module = PyType_GetModuleByToken( - Py_TYPE(self), examplemodule_slots); + PyObject *module = PyType_GetModuleByDef( + Py_TYPE(self), (PyModuleDef*)MOD_TOKEN); if (!module) { return NULL; } examplemodule_state *state = PyModule_GetState(module); - Py_DECREF(module); if (!state) { return NULL; } - return PyUnicode_FromFormat("<%T object; module value = %d>", - self, state->value); + return PyUnicode_FromFormat("", + state->value); } static PyType_Spec exampletype_spec = { @@ -105,13 +112,17 @@ examplemodule_exec(PyObject *module) { PyDoc_STRVAR(examplemodule_doc, "Example extension."); -static PyModuleDef_Slot examplemodule_slots[] = { - {Py_mod_name, "examplemodule"}, - {Py_mod_doc, (char*)examplemodule_doc}, - {Py_mod_methods, examplemodule_methods}, - {Py_mod_state_size, (void*)sizeof(examplemodule_state)}, - {Py_mod_exec, (void*)examplemodule_exec}, - {0} +PyABIInfo_VAR(abi_info); + +static PySlot examplemodule_slots[] = { + PySlot_STATIC_DATA(Py_mod_abi, &abi_info), + PySlot_STATIC_DATA(Py_mod_name, "examplemodule"), + PySlot_STATIC_DATA(Py_mod_doc, (char*)examplemodule_doc), + PySlot_STATIC_DATA(Py_mod_methods, examplemodule_methods), + PySlot_SIZE(Py_mod_state_size, sizeof(examplemodule_state)), + PySlot_FUNC(Py_mod_exec, examplemodule_exec), + PySlot_STATIC_DATA(Py_mod_token, MOD_TOKEN), + PySlot_END }; // Avoid "implicit declaration of function" warning: diff --git a/peps/pep-0793/shim.c b/peps/pep-0793/shim.c index 4e6b67e805e..017a1361353 100644 --- a/peps/pep-0793/shim.c +++ b/peps/pep-0793/shim.c @@ -1,57 +1,91 @@ -#include // memset +#include -PyMODINIT_FUNC PyInit_examplemodule(void); +// Hack: Restore old definition of Py_TYPE +#undef Py_TYPE +#define Py_TYPE(OBJ) (((PyObject*)OBJ)->ob_type) +// PyModuleDef, also reused as module token static PyModuleDef module_def_and_token; +#define MOD_TOKEN (&module_def_and_token) + +#include "examplemodule.c" + +extern PySlot *PyModExport_examplemodule(void); + +PyMODINIT_FUNC PyInit_examplemodule(void); PyMODINIT_FUNC PyInit_examplemodule(void) { - PyModuleDef_Slot *slot = PyModExport_examplemodule(); - if (module_def_and_token.m_name) { // Take care to only set up the static PyModuleDef once. // (PyModExport might theoretically return different data each time.) return PyModuleDef_Init(&module_def_and_token); } - int copying_slots = 1; - for (/* slot set above */; slot->slot; slot++) { - switch (slot->slot) { + + static PyModuleDef_Slot module_slots[5] = {{0}}; + module_def_and_token.m_slots = module_slots; + int current_m_slot = 0; + + PySlot *slot = PyModExport_examplemodule(); + + for (/* slot set above */; slot->sl_id; slot++) { + switch (slot->sl_id) { // Set PyModuleDef members from slots. These slots must come first. -# define COPYSLOT_CASE(SLOT, MEMBER, TYPE) \ - case SLOT: \ - if (!copying_slots) { \ - PyErr_SetString(PyExc_SystemError, \ - #SLOT " must be specified earlier"); \ - goto error; \ - } \ - module_def_and_token.MEMBER = (TYPE)(slot->value); \ - break; \ - ///////////////////////////////////////////////////////////////// - COPYSLOT_CASE(Py_mod_name, m_name, char*) - COPYSLOT_CASE(Py_mod_doc, m_doc, char*) - COPYSLOT_CASE(Py_mod_state_size, m_size, Py_ssize_t) - COPYSLOT_CASE(Py_mod_methods, m_methods, PyMethodDef*) - COPYSLOT_CASE(Py_mod_state_traverse, m_traverse, traverseproc) - COPYSLOT_CASE(Py_mod_state_clear, m_clear, inquiry) - COPYSLOT_CASE(Py_mod_state_free, m_free, freefunc) +# define COPYSLOT_CASE(SLOT, DEF_MEMBER, SL_MEMBER, TYPE) \ + case SLOT: \ + if (slot->sl_flags & PySlot_INTPTR) { \ + module_def_and_token.DEF_MEMBER = (TYPE)(slot->sl_ptr); \ + } else { \ + module_def_and_token.DEF_MEMBER = (TYPE)(slot->SL_MEMBER);\ + } \ + break; \ + /////////////////////////////////////////////////////////////////// + COPYSLOT_CASE(Py_mod_name, m_name, sl_ptr, char*) + COPYSLOT_CASE(Py_mod_doc, m_doc, sl_ptr, char*) + COPYSLOT_CASE(Py_mod_state_size, m_size, sl_size, Py_ssize_t) + COPYSLOT_CASE(Py_mod_methods, m_methods, sl_ptr, PyMethodDef*) + COPYSLOT_CASE(Py_mod_state_traverse, m_traverse, sl_func, traverseproc) + COPYSLOT_CASE(Py_mod_state_clear, m_clear, sl_func, inquiry) + COPYSLOT_CASE(Py_mod_state_free, m_free, sl_func, freefunc) + COPYSLOT_CASE(Py_mod_slots, m_slots, sl_ptr, PyModuleDef_Slot*) +# undef COPYSLOT_CASE + case Py_mod_create: + case Py_mod_exec: + case Py_mod_multiple_interpreters: + case Py_mod_gil: + int old_slot_id = (int)slot->sl_id; + if (old_slot_id > 83) { + // Hack: slots were renumbered; use old IDs here + old_slot_id -= 83; + } + module_slots[current_m_slot].slot = old_slot_id; + module_slots[current_m_slot].value = slot->sl_ptr; + current_m_slot++; + if (current_m_slot >= 4) { + PyErr_SetString(PyExc_SystemError, + "Too many slots for array"); + goto error; + } + break; case Py_mod_token: // With PyInit_, the PyModuleDef is used as the token. - if (slot->value != &module_def_and_token) { + if (slot->sl_ptr != &module_def_and_token) { PyErr_SetString(PyExc_SystemError, "Py_mod_token must be set to " "&module_def_and_token"); goto error; } break; + case Py_mod_abi: + // ABI checking skipped here + break; default: - // The remaining slots become m_slots in the def. - // (`slot` now points to the "rest" of the original - // zero-terminated array.) - if (copying_slots) { - module_def_and_token.m_slots = slot; + if (!(slot->sl_flags & PySlot_OPTIONAL)) { + PyErr_Format(PyExc_SystemError, + "Unknown slot ID %d.", (int)slot->sl_id); + goto error; } - copying_slots = 0; break; } } @@ -63,6 +97,6 @@ PyInit_examplemodule(void) return PyModuleDef_Init(&module_def_and_token); error: - memset(&module_def_and_token, 0, sizeof(module_def_and_token)); + module_def_and_token.m_name = NULL; return NULL; } diff --git a/peps/pep-0803.rst b/peps/pep-0803.rst index 4b51e47aca0..36174632ec4 100644 --- a/peps/pep-0803.rst +++ b/peps/pep-0803.rst @@ -2,7 +2,7 @@ PEP: 803 Title: "abi3t": Stable ABI for Free-Threaded Builds Author: Petr Viktorin , Nathan Goldbaum Discussions-To: https://discuss.python.org/t/106181 -Status: Accepted +Status: Final Type: Standards Track Requires: 703, 793, 697 Created: 19-Aug-2025 @@ -12,6 +12,8 @@ Post-History: `08-Sep-2025 `__, `16-Feb-2026 `__, Resolution: `30-Mar-2026 `__ +.. canonical-doc:: :ref:`py3.15:stable-abi` and :ref:`py3.15:abi3t-migration-howto` + Abstract ======== @@ -330,7 +332,7 @@ significant changes to their code. Projects that cannot do this (yet) can continue using ``abi3``, and compile the same source for specific versions of free-threaded builds. -(Note that the APIs removed in ``api3t`` still are usable when compiling for +(Note that the APIs removed in ``abi3t`` still are usable when compiling for a specific version, including 3.15t.) See a Rejected Ideas sections for an alternative: @@ -462,6 +464,11 @@ set the configuration macros automatically in the following situations: (This means that CPython can continue to use the ``Py_LIMITED_API`` macro internally to select which APIs are available.) +Also, if ``Py_TARGET_ABI3T`` is defined, then ``Python.h`` will make sure +that ``Py_GIL_DISABLED`` is defined as well. +Users may check this macro to enable free-threading-specific code like +extra locking. + Opaque PyObject --------------- @@ -556,8 +563,11 @@ tools, or human mistakes, in the form of a new *module slot*, ``Py_mod_abi``, containing basic ABI information. This information will be checked when a module is loaded, and incompatible extensions will be rejected. -The specifics are left to the C API working group -(see `capi-workgroup issue 72 `__). +The specifics are left to the C API working group. +(See `capi-workgroup issue 72 `__, which was implemented well before this PEP was finalized. +Additionally, a ``PyABIInfo_FREETHREADING_AGNOSTIC`` +flag for ``PyABIInfo.flags`` will be added to signal compatibility with both +``abi3`` and ``abi3t``.) This slot will become *mandatory* with the new export hook added in :pep:`793`. @@ -659,9 +669,8 @@ For example, a wheel tagged ``cp315-abi3.abi3t`` will be compatible with .. _Python tag: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#python-tag It is discouraged, but possible, to compile extensions compatible with *only* -``abi3t`` (by defining only :samp:`Py_TARGET_ABI3T={v}`, building with -GIL-enabled CPython, and tagging the resulting wheel with ``abi3t`` rather -than ``abi3.abi3t``). +``abi3t`` (by defining only :samp:`Py_TARGET_ABI3T={v}` and tagging the +resulting wheel with ``abi3t`` rather than ``abi3.abi3t``). This will limit the result to free-threaded interpreters only. Its is also possible to build ``abi3t`` extensions compatible with CPython 3.14 @@ -677,9 +686,10 @@ Implementing this PEP will make it possible to build extensions that can be successfully loaded on free-threaded Python, but not necessarily ones that are thread-safe without a GIL. -Limited API to allow thread-safety without a GIL -- presumably -``PyCriticalSection`` and similar -- will be added via the C API working group, -or in a follow-up PEP. +Limited API to allow thread-safety without a GIL will be added via the +C API working group, or in a follow-up PEP. +(Note: :external+py3.15:c:type:`PyCriticalSection` API was added to 3.15 in +`C API WG issue 100 `__.) Backwards and Forwards Compatibility @@ -872,11 +882,13 @@ This PEP combines several pieces, implemented individually: `python/cpython#137212 `__. - A check for older ``abi3`` was implemented in GitHub pull request `python/cpython#137957 `__. -- For wheel tag handling in installers, a draft pull request is at +- The ``packaging`` project implemented wheel tag handling in installers in `pypa/packaging/pull#1099 `__. -- For build tools, several individual draft pull requests are open; +- For build tools, several individual pull requests were made; contact Nathan for details. -- A porting guide is not yet written. + +After this PEP was accepted, the implementation was tracked in +`CPython issue 146636 `__. .. _pep803-rejected-ideas: diff --git a/peps/pep-0820.rst b/peps/pep-0820.rst index e7fe352c4fa..c44728987da 100644 --- a/peps/pep-0820.rst +++ b/peps/pep-0820.rst @@ -2,13 +2,15 @@ PEP: 820 Title: PySlot: Unified slot system for the C API Author: Petr Viktorin Discussions-To: https://discuss.python.org/t/105552 -Status: Accepted +Status: Final Type: Standards Track Created: 19-Dec-2025 Python-Version: 3.15 Post-History: `06-Jan-2026 `__ Resolution: `23-Apr-2025 `__ +.. canonical-doc:: :ref:`py3.15:capi-slots` + .. highlight:: c @@ -135,7 +137,7 @@ This proposal adds API to create classes and modules from arrays of slots, which can be specified as C literals using macros, like this:: static PySlot myClass_slots[] = { - PySlot_STATIC(tp_name, "mymod.MyClass"), + PySlot_STATIC_DATA(tp_name, "mymod.MyClass"), PySlot_SIZE(tp_extra_basicsize, sizeof(struct myClass)), PySlot_FUNC(tp_repr, myClass_repr), PySlot_INT64(tp_flags, Py_TPFLAGS_DEFAULT | Py_TPFLAGS_MANAGED_DICT), @@ -256,8 +258,8 @@ and only use the “new” slots if they need any new features. Fixed-width integers --------------------- -This proposal uses fixed-width integers (``uint16_t``) for slot IDs and -flags. +This proposal uses fixed-width integers for slot IDs (``uint16_t``) and +flags (``uint64_t``). With the C ``int`` type, using more than 16 bits would not be portable, but it would silently work on common platforms. Using ``int`` but avoiding values over ``UINT16_MAX`` wastes 16 bits @@ -269,7 +271,7 @@ Memory layout On common 64-bit platforms, we can keep the size of the new struct the same as the existing ``PyType_Slot`` and ``PyModuleDef_Slot``. (The existing -struct waste 6 out of 16 bytes due to ``int`` portability and padding; +structs waste 6 out of 16 bytes due to ``int`` portability and padding; this proposal puts some of those bits to use for new features.) On 32-bit platforms, this proposal calls for the same layout as on 64-bit, doubling the size compared to the existing structs (from 8 bytes to 16). @@ -422,6 +424,9 @@ Flags *array* of ``PyMemberDef`` structures, then the entire array, as well as the ``name`` and ``doc`` strings in its elements, must be static and constant. + This flag will be required for slots that need static data + (``Py_mod_methods``, ``Py_tp_methods``, ``Py_tp_members``, ``Py_tp_getset``). + - ``PySlot_INTPTR``: The data is stored in ``sl_ptr``, and must be cast to the appropriate type. @@ -482,7 +487,8 @@ Two more slots will allow similar nesting for existing slot structures: - ``Py_mod_slots`` for an array of ``PyModuleDef_Slot`` Each ``PyType_Slot`` in the array will be converted to -``(PySlot){.sl_id=slot, .sl_flags=PySlot_INTPTR, .sl_ptr=func}``, +``(PySlot){.sl_id=slot, .sl_flags=PySlot_INTPTR, .sl_ptr=func}`` +(with ``PySlot_STATIC`` to ``sl_flags`` added for slots that require it), and similar with ``PyModuleDef_Slot``. In the initial implementation, nesting depth will be limited to 5 levels. @@ -614,7 +620,7 @@ Backwards Compatibility This PEP proposes to change API that was already released in alpha versions of Python 3.15. This will inconvenience early adopters of that API, but -- as long as the -PEP is accepted and implemented before the first bety -- this change is within +PEP is accepted and implemented before the first beta -- this change is within the letter and spirit of our backwards compatibility policy. Renumbering of slots is done in a backwards-compatible way. @@ -643,8 +649,8 @@ Adjust the "Extending and Embedding" tutorial to use this. Reference Implementation ======================== -Draft implementation is available as `pull request #37 in the author's fork -`__. +After this PEP was accepted, the implementation was merged in +`CPython issue 149044 `__. Rejected Ideas @@ -743,7 +749,13 @@ for substantial input on this iteration of the proposal. Change History ============== +* 17-Jun-2026 + + - Require explicit ``PySlot_STATIC`` flag for slots that need static data. + - PEP marked final. + * 24-Apr-2026 + - Limit deprecation for NULL and repeated slots to the new API. - PEP is accepted