💬 Trending Now: pep - Fox News Update 💬 Trending Now: pep - Fox News Update 💬 Trending Now: pep - Fox News Update

Python Enhancement Proposals

PEP 780 – ABI features as environment markers

PEP 780 – ABI features as environment markers

Author:
Klaus Zimmermann <klaus_zimmermann at gmx.de>, Ralf Gommers <ralf.gommers at gmail.com>
Sponsor:
Lysandros Nikolaou <lisandrosnik at gmail.com>
Discussions-To:
Discourse thread
Status:
Draft
Type:
Standards Track
Topic:
Packaging
Created:
21-Mar-2025
Post-History:
05-Aug-2024, 26-Mar-2025, 06-Nov-2026

Table of Contents

Abstract

This PEP defines the use of information about the interpreter ABI as environment markers for project dependencies through a new sys_abi_info environment marker variable. PEP 508 (later moved to Dependency specifiers) introduced environment markers to specify dependencies based on rules that describe when the dependency should be used. This PEP extends environment markers to allow specifying dependencies based on specific information about the ABI of the Python interpreter. For this, it defines a set of ABI Features and specifies how they are made available for environment markers via a new marker variable, sys_abi_info.

Use of sys_abi_info in Requires-Dist metadata is introduced with Core Metadata version 2.7.

Motivation

In 2015, PEP 508 established environment markers to specify dependencies based on environment conditions. The development of free-threaded CPython [1] has underlined the need for an environment marker to discriminate between different ABI features that the interpreter was built with. For example, currently there is no way to distinguish between a GIL-enabled and a free-threaded CPython interpreter with an environment marker. This leads to real world issues for the adoption of free-threading and its incremental rollout. When a Python package is being made compatible with free-threaded CPython, it also needs all its build and runtime dependencies to be compatible. Capturing the first version of a dependency that is compatible precisely in metadata is currently not possible, and increasing the minimum version of a dependency also for the GIL-enabled build is usually undesirable since it unnecessarily limits compatibility between packages.

At the time of writing of the first version of this PEP, concrete examples of such issues were discussed in the Environment marker for free-threading Discourse thread:

  • Cython had experimental support for free-threading only in its master branch, and was used by many projects already publishing cp313t wheels. Picking up the wrong Cython version caused obscure build failures and runtime crashes. It would have been beneficial if the metadata could have expressed that (see Examples).
  • CFFI did not yet support free-threading, and for a significant period of time, it looked likely that support would first be added in a fork. Many projects depend on cffi and could have depended on a fork for free-threading only; however, depending on a fork for >=3.13 or for all Python versions seemed like a much larger ask, and more disruptive for distribution packagers.

Cython 3.1.0 and CFFI 2.0.0 have since added free-threading support, but the same principle remains relevant as other projects further up the stack add support. The free-threading rollout is an ongoing multi-year effort, and an environment marker for free-threading will make that rollout significantly easier.

Another important ABI feature that is not yet covered by environment markers is the bitness of the interpreter. In most cases, the sys_platform or platform_system markers are enough, because there is only a single bitness in use per platform. This is not the case on Windows however: both 32-bit and 64-bit Python interpreters are widely used on x86-64 Windows. Not being able to distinguish between the two may be relevant for packages that provide compiled extensions. For example, SciPy does not provide win32 wheels (it isn’t able to due to the lack of a suitable 32-bit compiler toolchain with Fortran support). Those wheels lacking can be awkward especially for projects where SciPy is an optional dependency only. In that case, it would be useful to be able to specify that SciPy is required unless the interpreter is a 32-bit one on Windows (see Examples), to avoid failed from-source installations due to the missing wheels.

Enabling consistent dependency metadata across a package release is another motivation for this PEP. The PEP 825 investigation of metadata consistency identified apache-beam’s omission of pyarrow on 32-bit Windows as the only remaining real-world case in that survey where existing markers could not express the required dependency conditions. The bitness information exposed by sys_abi_info closes that gap, allowing the condition to be expressed in the same dependency metadata in the sdist and every wheel of the release.

Closing this gap also enables future requirements for consistent dependency metadata across release artifacts. Resolvers can then use one artifact’s dependency metadata for the release, evaluating markers for each target environment, rather than inspecting every wheel separately. Tools such as uv already expect this consistency.

Rationale

The intention of this PEP is to introduce its core features with minimal impact on the existing ecosystem. The existing grammar proposed in PEP 508 lends itself to a straightforward extension to include the new environment marker.

A Forward Looking View on Free-Threaded Python

PEP 703, the accepted proposal for free threading, states that the rollout of free-threaded Python should be gradual, which has been clarified by the Python Steering Council in the PEP 703 acceptance post to mean a three stage process over multiple releases. It is therefore important to make sure that the mechanisms in this PEP are useable for Python interpreters where either free-threading or non-free-threading could be the default or the only option.

When the first version of this PEP was written, free-threaded Python was in Phase I: experimental phase. The proposed environment markers were needed to help with the transition as package authors gradually added support.

Since Python 3.14, free-threaded Python is officially supported, entering Phase II: Supported-but-not-default phase, as described in PEP 779. As the number of packages with support increases, there remains a need for environment markers to help with this ongoing transition.

As free-threaded Python enters into Phase III: Default phase, the need for the environment markers will decrease, though at this point it is not clear that the GIL-enabled Python will be completely phased out (it may remain available as a non standard build option). If it persists, the inverse need for the ABI feature detection may arise.

Indeed, in all three phases it may be necessary for package authors to choose specific versions of their dependencies based on the ABI features, with a shift from GIL-enabled as default to free-threading as default over time.

The ABI features are designed with this in mind to guarantee usefulness and simplicity for the foreseeable future in a changing Python ecosystem.

Relation to Other PEPs

PEP 751, which standardized pylock.toml, introduced set-valued environment markers for extras and dependency groups in lock files. This PEP applies those set semantics to ABI features through sys_abi_info, making them available in dependency specifications as well as lock files.

PEP 825 also introduces new environment markers as part of its Wheel Variants design. The considerations in Backwards Compatibility and the analysis in the tool behavior appendix apply equally to those markers when encountered by tools that do not support them.

Specification

ABI Features

ABI features are intrinsic properties of the Python interpreter. However, not all features are equally applicable to all Python interpreters or Python versions. For example, CPython introduced free-threading support in Python 3.13. Other implementations do not currently support free-threading, so free_threaded MUST be False for them. The bitness of the interpreter, i.e. the number of bits in a pointer, is relevant for all interpreters.

Tools MUST provide the ABI features listed below for all Python implementations, using sys.abi_info where available and the rules in Determining ABI Features on Older Interpreters otherwise:

pointer_bits
The bitness of the interpreter, that is, the number of bits in a pointer, typically 32 or 64 [2].
free_threaded
If the Python interpreter was built with free-threading support, free_threaded MUST be True, otherwise it MUST be False. This value reflects the build configuration, not whether the GIL is currently enabled. It is unaffected by -X gil, PYTHON_GIL, or changes to the GIL state at runtime.
debug
Indicates whether the interpreter was built in debug mode, corresponding to the sys.abi_info.debug field.
byteorder
Indicates the byte order of the interpreter. The value MUST be either "little" or "big".

For implementations without the corresponding sys.abi_info fields, the values of free_threaded and debug are determined by the rules in Determining ABI Features on Older Interpreters, including the False defaults for non-CPython implementations.

The sys_abi_info Environment Marker

To make ABI features available in dependency specifications, a new environment marker variable, sys_abi_info, is added to the format of dependency specifiers. It is built to reflect the value of sys.abi_info, which is a new attribute of the sys module introduced in Python 3.15 as a simple namespace. To represent its fields in the environment marker variable, they are expressed as a frozen set of strings of the form f"{name}::{value}", where name is the name of the field and value is the string representation of its value.

The sys_abi_info Environment Variable

To do this, we need to extend the grammar laid out in PEP 508 and maintained in the Dependency specifiers and document the possible values, to include the sys_abi_info marker variable by augmenting the definition of env_var as follows:

env_var       = ('python_version' | 'python_full_version' |
                 'os_name' | 'sys_platform' | 'platform_release' |
                 'platform_system' | 'platform_version' |
                 'platform_machine' | 'platform_python_implementation' |
                 'implementation_name' | 'implementation_version' |
                 'sys_abi_info' |
                 'extra' | 'extras' | 'dependency_groups' # ONLY when defined by a containing layer
                 )

This makes sys_abi_info available as a variable in marker expressions.

The Values of sys_abi_info

Like the grammar, also the overview table of environment markers in Dependency specifiers is augmented to add the following row:

Marker Python equivalent Sample values
sys_abi_info sys.abi_info {'free_threaded::True', 'debug::False', 'pointer_bits::64', 'byteorder::little'}, {'free_threaded::False', 'debug::True', 'pointer_bits::32', 'byteorder::little'}, {'free_threaded::False', 'debug::False', 'pointer_bits::64', 'byteorder::little'}

With these additions, ABI features can be used in dependency specifications via the in operator to test for the presence of a feature-value pair, or the not in operator to test for its absence.

This allows for marker expressions such as:

"free_threaded::True" in sys_abi_info
"pointer_bits::32" not in sys_abi_info and "debug::True" in sys_abi_info

Integer-valued features support exact-value tests, not numerical comparisons. This is sufficient for the bitness use case: excluding 32-bit Windows requires testing for "pointer_bits::32", without enumerating other possible pointer widths. An unfamiliar value, such as 128, remains a distinct string; absence of "pointer_bits::32" does not imply a 64-bit interpreter.

Adding new ABI features does not require changes to the marker grammar, because the feature names and values are encoded as strings. If the tested string is absent from sys_abi_info, in evaluates to false and not in evaluates to true. Package authors can therefore choose the default behavior for interpreters that do not provide a new feature. For example, "new_feature::True" in sys_abi_info requires the feature to be explicitly true, while "new_feature::False" not in sys_abi_info also evaluates to true when the feature is absent.

ABI feature values MUST be unambiguously representable as strings. Currently, all ABI features are either booleans, integers, or strings, which all satisfy this requirement. This is expected to remain the case. When constructing sys_abi_info, packaging tools MUST ignore fields whose values have other data types.

Core Metadata

Requires-Dist is a Core Metadata field whose value syntax is defined by the Dependency Specifiers specification. Because this PEP changes what is considered valid in that field, it updates Core Metadata to version 2.7, following the precedent of PEP 685.

Build tools serializing a Requires-Dist entry whose environment marker references sys_abi_info MUST emit Core Metadata version 2.7 or later. Build tools MAY continue to emit an earlier Core Metadata version when sys_abi_info is not used in Requires-Dist and no other metadata feature requires a newer version.

Publishing tools SHOULD report an error when a distribution uses sys_abi_info in Requires-Dist while declaring a Core Metadata version earlier than 2.7. Index servers SHOULD reject newly uploaded distributions with this inconsistency.

This requirement applies only when the marker is serialized into Core Metadata. Uses of sys_abi_info in other dependency-specifier contexts, such as build-system.requires, requirements files, or pylock.toml, do not by themselves require or imply a Core Metadata version.

Determining ABI Features on Older Interpreters

The features defined by this PEP describe properties that were already available before sys.abi_info was introduced in Python 3.15. Tools MUST provide them even when that attribute, or an applicable field, is absent. When a field is available in sys.abi_info, tools MUST use its value. Otherwise, the following rules define the values for the target interpreter; implementations MAY use equivalent methods to obtain the same values.

pointer_bits
8 * struct.calcsize("P").
byteorder
sys.byteorder.
free_threaded
On CPython, bool(sysconfig.get_config_var("Py_GIL_DISABLED")). An undefined configuration variable is treated as false. This detects build-time support, independently of the runtime GIL state. On other implementations, the value MUST be False.
debug
On CPython, bool(sysconfig.get_config_var("Py_DEBUG")) when the configuration variable is defined. Otherwise, the value is true if hasattr(sys, "gettotalrefcount") is true or "_d.pyd" is present in importlib.machinery.EXTENSION_SUFFIXES, and false otherwise. The latter check covers Windows builds that do not expose Py_DEBUG. On other implementations, the value MUST be False.

The resulting values are encoded as "name::value" strings in the same way as values obtained from sys.abi_info; false boolean values are included, not omitted. Tools do not need to add sys.abi_info to the interpreter.

Future attributes exposed by sys.abi_info are included when their values have a supported type. Inferring those attributes on interpreters where they are absent requires additional rules in the Dependency Specifiers specification. Unless such rules are defined, tools MUST omit the missing attributes rather than infer a value. In particular, an absent boolean attribute is distinct from one whose value is false.

Handling Unsupported Marker Fields

The Dependency Specifiers specification states that references to unknown marker fields SHOULD render a package version ineligible for installation or inclusion in a locked dependency tree. This PEP clarifies that rule as follows.

Consumers do not need to successfully parse an unknown marker field to implement this behavior. When a locking or installation tool cannot interpret published dependency metadata because a marker field is unsupported, it SHOULD treat the containing package version as ineligible and SHOULD continue considering other versions where possible. Such tools MAY conservatively reject a package version when any of its published dependency specifications cannot be parsed, without distinguishing unsupported marker fields from malformed requirements.

When determining dependencies for installation or inclusion in a locked dependency tree, tools MUST NOT recover by dropping an unsupported marker, discarding the dependency that contains it, or assuming a truth value for its expression. Such recovery can produce an apparently successful installation with incorrect dependencies. This does not prevent static analysis tools from ignoring markers when collecting all potential dependencies. If no eligible candidate remains, installation and locking tools SHOULD report a descriptive failure explaining the unsupported metadata and, where possible, suggesting an upgrade of the consuming tool.

This clarification concerns interpreting distribution dependency metadata, including metadata in wheels or successfully generated from an sdist. It does not require tools to recover from failures while preparing an sdist or generating its metadata, as discussed in Backwards Compatibility.

Examples

To require a pre-release of Cython only for a free-threaded Python interpreter, the following dependency specification can be used:

cython >3.1.0a1; "free_threaded::True" in sys_abi_info
cython ==3.0.*; "free_threaded::False" in sys_abi_info

To require SciPy unless on a 32-bit interpreter on Windows, the following dependency specification can be used:

scipy; platform_system != "Windows" or "pointer_bits::32" not in sys_abi_info

To require NumPy only for a free-threaded interpreter with debugging capabilities, the following dependency can be used:

numpy; "free_threaded::True" in sys_abi_info and "debug::True" in sys_abi_info

Require Free-Threaded Fork of PyYAML

PyYAML initially decided not to support free-threading in its mainline distribution before wider testing of free-threaded Python in the real world. To facilitate this testing, a free-threaded fork, PyYAML-ft, was created. To require this fork, for a free-threaded Python interpreter, without forcing it also on the GIL-enabled interpreter, the following dependency specification can be used:

pyyaml-ft; "free_threaded::True" in sys_abi_info
pyyaml; "free_threaded::True" not in sys_abi_info

Note that in the absence of the features described in this PEP, early adopting projects have had to resort to using the fork for all builds of a potentially free threading Python interpreter; typical dependencies then looked like:

pyyaml; python_version<"3.13"
pyyaml-ft; python_version>="3.13"

which is an undesirable side effect for GIL-enabled environments.

Backwards Compatibility

Introducing a new environment marker can affect tools that do not understand it. Packaging interoperability generally follows the principle of strict validation when producing or accepting new metadata and forward-compatible handling when consuming previously published metadata.

Build backends, publishing tools, and package indexes should understand and validate new metadata features before producing or accepting them. Consumers can ignore new optional information when it is unnecessary for the operation being performed. When new semantics are needed to determine dependencies or compatibility, consumers should instead consider the affected package version unsupported.

Core Metadata version 2.7 provides an explicit producer and publisher boundary for using sys_abi_info in Requires-Dist. It does not by itself make older consumers compatible with the new marker. Core Metadata permits consumers to process newer minor versions, so an older consumer may continue processing 2.7 metadata and encounter the unsupported marker. The fallback behavior described below is therefore still needed. Beyond the metadata version requirement, this PEP introduces no separate index activation date or waiting period.

Marker field names are part of a closed grammar. Tools that predate sys_abi_info may therefore report a parsing error when encountering a dependency containing it. An open-ended marker grammar is not needed to handle this safely: the resolver can treat the containing package version as ineligible and try other versions. This preserves correctness without requiring the consumer to understand the new marker. See Handling Unsupported Marker Fields for the specification clarification.

Not all existing tools implement this fallback behavior. For wheel dependency metadata, pip 24.0 aborts resolution on an unsupported marker, while pip 24.1 (Jun 2024) and later can skip the affected package version and select an older version. Poetry versions preceding the release that resolves poetry#11191 instead drop the entire dependency containing an unknown marker and keep the package version eligible. This can produce an apparently successful installation with a missing dependency, leading to runtime failures. It is therefore unsafe even when resolution succeeds.

Publishing a release that uses sys_abi_info in dependency metadata therefore drops compatibility with pip <24.1, and with Poetry (until Poetry implements a fix). Projects adopting the marker should make this compatibility change clear to their users.

Pinning a version or requesting an artifact directly may leave no eligible alternative, resulting in an installation failure.

Source distributions have additional limitations. An unsupported marker in build-system.requires can cause the frontend to fail before running the backend. An unsupported marker in project.dependencies can cause the backend to fail during metadata generation. Validation may occur as early as the backend hook that determines additional build requirements. These errors may abort installation rather than trigger fallback to an older release, including when installing from an index. This PEP does not require changes to that behavior: failure is an acceptable outcome for older tools encountering such source distributions. Installing a usable wheel does not require inspecting the sdist’s build requirements. The tool behavior appendix records these distinctions in detail.

Projects adopting sys_abi_info should account for these compatibility boundaries and the installation and locking tools their users rely on. Projects decide when to adopt the marker; this PEP specifies no minimum ecosystem support threshold. Once tools understand sys_abi_info, additional values in its set can be handled through the membership semantics specified above. Unknown marker fields must not be confused with absent or unknown ABI feature values.

Audit and Update Tools

A wide range of tools, including Dependabot and Tidelift, inspect dependency data in pyproject.toml, pylock.toml, and requirements.txt files and offer assisted or automated updates. Tools that interpret the meaning of sys_abi_info need explicit support for it. Tools consuming published distributions can remain correct by rejecting versions with unsupported dependency metadata, although this can limit which releases they consider. Static analysis and update workflows may lose functionality until their tools support the marker.

Security Implications

This PEP introduces new syntax for specifying dependency information in projects. However, it does not introduce newly specified mechanisms for handling or resolving dependencies.

It therefore does not carry security concerns other than those inherent in any tools which may already be used to install dependencies—i.e. malicious dependencies may be specified here, just as they may be specified in requirements.txt files.

How to Teach This

The use of environment markers is well established and communicated chiefly in Dependency specifiers. The new environment marker can be introduced in the same document. Additionally, both for package authors and users, free-threading specific guidance can be provided at the Python free-threading guide.

Reference Implementation

The reference implementation for the environment markers is available in a fork of the packaging library at Environment markers for ABI features.

A demonstration package is also available.

Since pip uses a vendored copy of packaging internally, we also provide a patched version of pip, which replaces the vendored packaging with the reference implementation linked above.

uv prototype

An independent prototype by Charlie Marsh is available in astral-sh/uv#21840. It adds preview support for ABI feature markers during dependency resolution and wheel coverage checks. The prototype implements an earlier draft of this PEP, using the marker name sys_abi_features and its earlier value format.

Rejected Ideas

Extension Mechanism

In an early discussion of the topic (Environment marker for free-threading), the idea of a general extension mechanism for environment markers was brought up. While it is appealing to forego a whole PEP process should the need for new environment markers arise in the future, there are two main challenges.

First, a completely dynamic mechanism would present difficulties for tools that rely on static analysis of dependency specifications.

This means that even if a dynamic mechanism were to be adopted, new environment markers would likely still need to be spelled out in a PEP.

Second, the introduction of a dynamic mechanism would require a more complex implementation in the packaging library, which would be a significant departure from the current approach.

Footnotes

Acknowledgements

Thanks to Filipe Laíns for the suggestion of the abi_features attribute and to Stephen Rosen for the Backwards Compatibility section of PEP 735, which served as a template for the corresponding section in this PEP.

Change History

  • 05-Nov-2026
    • Rewrote Backwards Compatibility and clarified how installation and locking tools should reject package versions with unsupported markers and consider alternatives. Prohibited recovery that changes dependency semantics, while allowing static analysis to collect potential dependencies. Documented the limitations of fallback for source builds.
    • Added the tool behavior appendix, documenting pip, uv, PDM, and Poetry behavior when encountering unknown markers.
    • Introduced Core Metadata version 2.7 for Requires-Dist entries using sys_abi_info, with producer, publishing tool, and index validation rules. Other dependency-specifier contexts do not require this version.
    • Specified how tools determine ABI features when sys.abi_info or its fields are unavailable, including false defaults for free_threaded and debug on non-CPython implementations. Defined how future missing attributes are handled.
    • Clarified that free_threaded describes the interpreter’s build configuration rather than its runtime GIL state.
    • Clarified exact-value membership tests, behavior for absent values, and support for boolean, integer, and string values. Required tools to ignore fields with other value types.
    • Added consistent dependency metadata across sdists and wheels as a motivation, referencing the PEP 825 investigation of 32-bit Windows. Related the compatibility guidance to PEP 825’s new environment markers.
    • Updated the free-threading rollout discussion and the relationship to PEP 751, and added the independent uv prototype under Reference Implementation.
    • Removed the Open Issues section, all open issues were addressed.

Appendices