PEP 780 – ABI features as environment markers
- Author:
- Klaus Zimmermann <klaus_zimmermann at gmx.de>, Ralf Gommers <ralf.gommers at gmail.com>
- Sponsor:
- Lysandros Nikolaou <lisandrosnik at gmail.com>
- Discussions-To:
- Discourse thread
- Status:
- Draft
- Type:
- Standards Track
- Topic:
- Packaging
- Created:
- 21-Mar-2025
- Post-History:
- 05-Aug-2024, 26-Mar-2025, 06-Nov-2026
Abstract
This PEP defines the use of information about the interpreter ABI as
environment markers for project dependencies through a new sys_abi_info
environment marker variable. PEP 508 (later moved to
Dependency specifiers) introduced environment markers to
specify dependencies based on rules that describe when the dependency should
be used. This PEP extends environment markers to allow specifying dependencies
based on specific information about the ABI of the Python interpreter. For
this, it defines a set of ABI Features and specifies how they are made
available for environment markers via a new marker
variable, sys_abi_info.
Use of sys_abi_info in Requires-Dist metadata is introduced with Core
Metadata version 2.7.
Motivation
In 2015, PEP 508 established environment markers to specify dependencies based on environment conditions. The development of free-threaded CPython [1] has underlined the need for an environment marker to discriminate between different ABI features that the interpreter was built with. For example, currently there is no way to distinguish between a GIL-enabled and a free-threaded CPython interpreter with an environment marker. This leads to real world issues for the adoption of free-threading and its incremental rollout. When a Python package is being made compatible with free-threaded CPython, it also needs all its build and runtime dependencies to be compatible. Capturing the first version of a dependency that is compatible precisely in metadata is currently not possible, and increasing the minimum version of a dependency also for the GIL-enabled build is usually undesirable since it unnecessarily limits compatibility between packages.
At the time of writing of the first version of this PEP, concrete examples of such issues were discussed in the Environment marker for free-threading Discourse thread:
- Cython had experimental support for free-threading only in its master
branch, and was used by many projects already publishing
cp313twheels. Picking up the wrong Cython version caused obscure build failures and runtime crashes. It would have been beneficial if the metadata could have expressed that (see Examples). - CFFI did not yet support free-threading, and for a significant period of
time, it looked likely that support would first be added in a fork. Many
projects depend on
cffiand could have depended on a fork for free-threading only; however, depending on a fork for >=3.13 or for all Python versions seemed like a much larger ask, and more disruptive for distribution packagers.
Cython 3.1.0 and CFFI 2.0.0 have since added free-threading support, but the same principle remains relevant as other projects further up the stack add support. The free-threading rollout is an ongoing multi-year effort, and an environment marker for free-threading will make that rollout significantly easier.
Another important ABI feature that is not yet covered by environment markers is
the bitness of the interpreter. In most cases, the sys_platform or
platform_system markers are enough, because there is only a single bitness
in use per platform. This is not the case on Windows however: both 32-bit and
64-bit Python interpreters are widely used on x86-64 Windows. Not being able to
distinguish between the two may be relevant for packages that provide compiled
extensions. For example, SciPy does not provide win32 wheels (it isn’t able
to due to the lack of a suitable 32-bit compiler toolchain with Fortran
support). Those wheels lacking can be awkward especially for projects where
SciPy is an optional dependency only. In that case, it would be useful to be
able to specify that SciPy is required unless the interpreter is a 32-bit one
on Windows (see Examples), to avoid failed
from-source installations due to the missing wheels.
Enabling consistent dependency metadata across a package release is another
motivation for this PEP. The PEP 825 investigation of metadata consistency identified apache-beam’s omission of
pyarrow on 32-bit Windows as the only remaining real-world case in that
survey where existing markers could not express the required dependency
conditions. The bitness information exposed by sys_abi_info closes that
gap, allowing the condition to be expressed in the same dependency metadata in
the sdist and every wheel of the release.
Closing this gap also enables future requirements for consistent dependency metadata across release artifacts. Resolvers can then use one artifact’s dependency metadata for the release, evaluating markers for each target environment, rather than inspecting every wheel separately. Tools such as uv already expect this consistency.
Rationale
The intention of this PEP is to introduce its core features with minimal impact on the existing ecosystem. The existing grammar proposed in PEP 508 lends itself to a straightforward extension to include the new environment marker.
A Forward Looking View on Free-Threaded Python
PEP 703, the accepted proposal for free threading, states that the rollout of free-threaded Python should be gradual, which has been clarified by the Python Steering Council in the PEP 703 acceptance post to mean a three stage process over multiple releases. It is therefore important to make sure that the mechanisms in this PEP are useable for Python interpreters where either free-threading or non-free-threading could be the default or the only option.
When the first version of this PEP was written, free-threaded Python was in Phase I: experimental phase. The proposed environment markers were needed to help with the transition as package authors gradually added support.
Since Python 3.14, free-threaded Python is officially supported, entering Phase II: Supported-but-not-default phase, as described in PEP 779. As the number of packages with support increases, there remains a need for environment markers to help with this ongoing transition.
As free-threaded Python enters into Phase III: Default phase, the need for the environment markers will decrease, though at this point it is not clear that the GIL-enabled Python will be completely phased out (it may remain available as a non standard build option). If it persists, the inverse need for the ABI feature detection may arise.
Indeed, in all three phases it may be necessary for package authors to choose specific versions of their dependencies based on the ABI features, with a shift from GIL-enabled as default to free-threading as default over time.
The ABI features are designed with this in mind to guarantee usefulness and simplicity for the foreseeable future in a changing Python ecosystem.
Relation to Other PEPs
PEP 751, which standardized
pylock.toml, introduced set-valued environment markers for extras and
dependency groups in lock files. This PEP applies those set semantics to ABI
features through sys_abi_info, making them available in dependency
specifications as well as lock files.
PEP 825 also introduces new environment markers as part of its Wheel Variants design. The considerations in Backwards Compatibility and the analysis in the tool behavior appendix apply equally to those markers when encountered by tools that do not support them.
Specification
ABI Features
ABI features are intrinsic properties of the Python interpreter.
However, not all features are equally applicable to all Python interpreters or
Python versions. For example, CPython introduced free-threading support in
Python 3.13. Other implementations do not currently support free-threading,
so free_threaded MUST be False for them. The bitness of the interpreter,
i.e. the number of bits in a pointer, is relevant for all interpreters.
Tools MUST provide the ABI features listed below for all Python implementations,
using sys.abi_info where available and the rules in
Determining ABI Features on Older Interpreters otherwise:
pointer_bits- The bitness of the interpreter, that is, the number of bits in a pointer,
typically
32or64[2]. free_threaded- If the Python interpreter was built with free-threading support,
free_threadedMUST beTrue, otherwise it MUST beFalse. This value reflects the build configuration, not whether the GIL is currently enabled. It is unaffected by-X gil,PYTHON_GIL, or changes to the GIL state at runtime. debug- Indicates whether the interpreter was built in debug mode, corresponding
to the
sys.abi_info.debugfield. byteorder- Indicates the byte order of the interpreter. The value MUST be either
"little"or"big".
For implementations without the corresponding sys.abi_info fields, the
values of free_threaded and debug are determined by the rules in
Determining ABI Features on Older Interpreters, including the False
defaults for non-CPython implementations.
The sys_abi_info Environment Marker
To make ABI features available in dependency specifications, a new environment
marker variable, sys_abi_info, is added to the format of dependency
specifiers. It is built to reflect the value of sys.abi_info, which is a
new attribute of the sys module introduced in Python 3.15 as a simple
namespace. To represent its fields in the environment marker variable, they are
expressed as a frozen set of strings of the form f"{name}::{value}", where
name is the name of the field and value is the string representation of
its value.
The sys_abi_info Environment Variable
To do this, we need to extend the grammar laid out in PEP 508 and maintained
in the Dependency specifiers and document the possible values,
to include the sys_abi_info marker variable by augmenting the definition of
env_var as follows:
env_var = ('python_version' | 'python_full_version' |
'os_name' | 'sys_platform' | 'platform_release' |
'platform_system' | 'platform_version' |
'platform_machine' | 'platform_python_implementation' |
'implementation_name' | 'implementation_version' |
'sys_abi_info' |
'extra' | 'extras' | 'dependency_groups' # ONLY when defined by a containing layer
)
This makes sys_abi_info available as a variable in marker expressions.
The Values of sys_abi_info
Like the grammar, also the overview table of environment markers in Dependency specifiers is augmented to add the following row:
| Marker | Python equivalent | Sample values |
|---|---|---|
sys_abi_info |
sys.abi_info |
{'free_threaded::True', 'debug::False', 'pointer_bits::64', 'byteorder::little'},
{'free_threaded::False', 'debug::True', 'pointer_bits::32', 'byteorder::little'},
{'free_threaded::False', 'debug::False', 'pointer_bits::64', 'byteorder::little'} |
With these additions, ABI features can be used in dependency specifications via
the in operator to test for the presence of a feature-value pair, or the
not in operator to test for its absence.
This allows for marker expressions such as:
"free_threaded::True" in sys_abi_info
"pointer_bits::32" not in sys_abi_info and "debug::True" in sys_abi_info
Integer-valued features support exact-value tests, not numerical comparisons.
This is sufficient for the bitness use case: excluding 32-bit Windows requires
testing for "pointer_bits::32", without enumerating other possible pointer
widths. An unfamiliar value, such as 128, remains a distinct string;
absence of "pointer_bits::32" does not imply a 64-bit interpreter.
Adding new ABI features does not require changes to the marker grammar,
because the feature names and values are encoded as strings. If the tested
string is absent from sys_abi_info, in evaluates to false and not in
evaluates to true. Package authors can therefore choose the
default behavior for interpreters that do not provide a new feature. For
example, "new_feature::True" in sys_abi_info requires the feature to be
explicitly true, while "new_feature::False" not in sys_abi_info also
evaluates to true when the feature is absent.
ABI feature values MUST be unambiguously representable as strings. Currently,
all ABI features are either booleans, integers, or strings, which all satisfy
this requirement. This is expected to remain the case. When constructing
sys_abi_info, packaging tools MUST ignore fields whose values have other
data types.
Core Metadata
Requires-Dist is a Core Metadata field
whose value syntax is defined by the Dependency Specifiers specification. Because this PEP changes what
is considered valid in that field, it updates Core Metadata to version 2.7,
following the precedent of PEP 685.
Build tools serializing a Requires-Dist entry whose environment marker
references sys_abi_info MUST emit Core Metadata version 2.7 or later.
Build tools MAY continue to emit an earlier Core Metadata version when
sys_abi_info is not used in Requires-Dist and no other metadata feature
requires a newer version.
Publishing tools SHOULD report an error when a distribution uses
sys_abi_info in Requires-Dist while declaring a Core Metadata version
earlier than 2.7. Index servers SHOULD reject newly uploaded distributions
with this inconsistency.
This requirement applies only when the marker is serialized into Core
Metadata. Uses of sys_abi_info in other dependency-specifier contexts,
such as build-system.requires, requirements files, or pylock.toml,
do not by themselves require or imply a Core Metadata version.
Determining ABI Features on Older Interpreters
The features defined by this PEP describe properties that were already
available before sys.abi_info was introduced in Python 3.15. Tools MUST
provide them even when that attribute, or an applicable field, is absent.
When a field is available in sys.abi_info, tools MUST use its value.
Otherwise, the following rules define the values for the target interpreter;
implementations MAY use equivalent methods to obtain the same values.
pointer_bits8 * struct.calcsize("P").byteordersys.byteorder.free_threaded- On CPython,
bool(sysconfig.get_config_var("Py_GIL_DISABLED")). An undefined configuration variable is treated as false. This detects build-time support, independently of the runtime GIL state. On other implementations, the value MUST beFalse. debug- On CPython,
bool(sysconfig.get_config_var("Py_DEBUG"))when the configuration variable is defined. Otherwise, the value is true ifhasattr(sys, "gettotalrefcount")is true or"_d.pyd"is present inimportlib.machinery.EXTENSION_SUFFIXES, and false otherwise. The latter check covers Windows builds that do not exposePy_DEBUG. On other implementations, the value MUST beFalse.
The resulting values are encoded as "name::value" strings in the same way
as values obtained from sys.abi_info; false boolean values are included,
not omitted. Tools do not need to add sys.abi_info to the interpreter.
Future attributes exposed by sys.abi_info are included when their values
have a supported type. Inferring those attributes on interpreters where they
are absent requires additional rules in the Dependency Specifiers
specification. Unless such rules are defined, tools MUST omit the missing
attributes rather than infer a value. In particular, an absent boolean
attribute is distinct from one whose value is false.
Handling Unsupported Marker Fields
The Dependency Specifiers specification states that references to unknown marker fields SHOULD render a package version ineligible for installation or inclusion in a locked dependency tree. This PEP clarifies that rule as follows.
Consumers do not need to successfully parse an unknown marker field to implement this behavior. When a locking or installation tool cannot interpret published dependency metadata because a marker field is unsupported, it SHOULD treat the containing package version as ineligible and SHOULD continue considering other versions where possible. Such tools MAY conservatively reject a package version when any of its published dependency specifications cannot be parsed, without distinguishing unsupported marker fields from malformed requirements.
When determining dependencies for installation or inclusion in a locked dependency tree, tools MUST NOT recover by dropping an unsupported marker, discarding the dependency that contains it, or assuming a truth value for its expression. Such recovery can produce an apparently successful installation with incorrect dependencies. This does not prevent static analysis tools from ignoring markers when collecting all potential dependencies. If no eligible candidate remains, installation and locking tools SHOULD report a descriptive failure explaining the unsupported metadata and, where possible, suggesting an upgrade of the consuming tool.
This clarification concerns interpreting distribution dependency metadata, including metadata in wheels or successfully generated from an sdist. It does not require tools to recover from failures while preparing an sdist or generating its metadata, as discussed in Backwards Compatibility.
Examples
To require a pre-release of Cython only for a free-threaded Python interpreter, the following dependency specification can be used:
cython >3.1.0a1; "free_threaded::True" in sys_abi_info
cython ==3.0.*; "free_threaded::False" in sys_abi_info
To require SciPy unless on a 32-bit interpreter on Windows, the following dependency specification can be used:
scipy; platform_system != "Windows" or "pointer_bits::32" not in sys_abi_info
To require NumPy only for a free-threaded interpreter with debugging capabilities, the following dependency can be used:
numpy; "free_threaded::True" in sys_abi_info and "debug::True" in sys_abi_info
Require Free-Threaded Fork of PyYAML
PyYAML initially decided not to support free-threading in its mainline distribution before wider testing of free-threaded Python in the real world. To facilitate this testing, a free-threaded fork, PyYAML-ft, was created. To require this fork, for a free-threaded Python interpreter, without forcing it also on the GIL-enabled interpreter, the following dependency specification can be used:
pyyaml-ft; "free_threaded::True" in sys_abi_info
pyyaml; "free_threaded::True" not in sys_abi_info
Note that in the absence of the features described in this PEP, early adopting projects have had to resort to using the fork for all builds of a potentially free threading Python interpreter; typical dependencies then looked like:
pyyaml; python_version<"3.13"
pyyaml-ft; python_version>="3.13"
which is an undesirable side effect for GIL-enabled environments.
Backwards Compatibility
Introducing a new environment marker can affect tools that do not understand it. Packaging interoperability generally follows the principle of strict validation when producing or accepting new metadata and forward-compatible handling when consuming previously published metadata.
Build backends, publishing tools, and package indexes should understand and validate new metadata features before producing or accepting them. Consumers can ignore new optional information when it is unnecessary for the operation being performed. When new semantics are needed to determine dependencies or compatibility, consumers should instead consider the affected package version unsupported.
Core Metadata version 2.7 provides an explicit producer and publisher boundary
for using sys_abi_info in Requires-Dist. It does not by itself make
older consumers compatible with the new marker. Core Metadata permits
consumers to process newer minor versions, so an older consumer may continue
processing 2.7 metadata and encounter the unsupported marker. The fallback
behavior described below is therefore still needed. Beyond the metadata version
requirement, this PEP introduces no separate index activation date or waiting
period.
Marker field names are part of a closed grammar. Tools that predate
sys_abi_info may therefore report a parsing error when encountering a
dependency containing it. An open-ended marker grammar is not needed to handle
this safely: the resolver can treat the containing package version as
ineligible and try other versions. This preserves correctness without requiring
the consumer to understand the new marker. See
Handling Unsupported Marker Fields for the specification clarification.
Not all existing tools implement this fallback behavior. For wheel dependency metadata, pip 24.0 aborts resolution on an unsupported marker, while pip 24.1 (Jun 2024) and later can skip the affected package version and select an older version. Poetry versions preceding the release that resolves poetry#11191 instead drop the entire dependency containing an unknown marker and keep the package version eligible. This can produce an apparently successful installation with a missing dependency, leading to runtime failures. It is therefore unsafe even when resolution succeeds.
Publishing a release that uses sys_abi_info in dependency metadata therefore
drops compatibility with pip <24.1, and with Poetry (until Poetry implements a
fix). Projects adopting the marker should make this compatibility change clear
to their users.
Pinning a version or requesting an artifact directly may leave no eligible alternative, resulting in an installation failure.
Source distributions have additional limitations. An unsupported marker in
build-system.requires can cause the frontend to fail before running the
backend. An unsupported marker in project.dependencies can cause the backend
to fail during metadata generation. Validation may occur as early as the
backend hook that determines additional build requirements. These errors may
abort installation rather than trigger fallback to an older release, including
when installing from an index. This PEP does not require changes to that
behavior: failure is an acceptable outcome for older tools encountering such
source distributions.
Installing a usable wheel does not require inspecting the sdist’s build
requirements. The tool behavior appendix records these distinctions in detail.
Projects adopting sys_abi_info should account for these compatibility
boundaries and the installation and locking tools their users rely on. Projects
decide when to adopt the marker; this PEP specifies no minimum ecosystem support
threshold.
Once tools understand sys_abi_info, additional values in its set can be
handled through the membership semantics specified above. Unknown marker
fields must not be confused with absent or unknown ABI feature values.
Audit and Update Tools
A wide range of tools, including Dependabot and Tidelift, inspect dependency
data in pyproject.toml, pylock.toml, and requirements.txt files and
offer assisted or automated updates. Tools that interpret the meaning of
sys_abi_info
need explicit support for it. Tools consuming published distributions can
remain correct by rejecting versions with unsupported dependency metadata,
although this can limit which releases they consider. Static analysis and
update workflows may lose functionality until their tools support the marker.
Security Implications
This PEP introduces new syntax for specifying dependency information in projects. However, it does not introduce newly specified mechanisms for handling or resolving dependencies.
It therefore does not carry security concerns other than those inherent in any
tools which may already be used to install dependencies—i.e. malicious
dependencies may be specified here, just as they may be specified in
requirements.txt files.
How to Teach This
The use of environment markers is well established and communicated chiefly in Dependency specifiers. The new environment marker can be introduced in the same document. Additionally, both for package authors and users, free-threading specific guidance can be provided at the Python free-threading guide.
Reference Implementation
The reference implementation for the environment markers is available in a fork
of the packaging library at Environment markers for ABI features.
A demonstration package is also available.
Since pip uses a vendored copy of packaging internally, we also provide
a patched version of pip, which replaces the vendored packaging with
the reference implementation linked above.
uv prototype
An independent prototype by Charlie Marsh is available in
astral-sh/uv#21840. It adds
preview support for ABI feature markers during dependency resolution and wheel
coverage checks. The prototype implements an earlier draft of this PEP, using
the marker name sys_abi_features and its earlier value format.
Rejected Ideas
Extension Mechanism
In an early discussion of the topic (Environment marker for free-threading), the idea of a general extension mechanism for environment markers was brought up. While it is appealing to forego a whole PEP process should the need for new environment markers arise in the future, there are two main challenges.
First, a completely dynamic mechanism would present difficulties for tools that rely on static analysis of dependency specifications.
This means that even if a dynamic mechanism were to be adopted, new environment markers would likely still need to be spelled out in a PEP.
Second, the introduction of a dynamic mechanism would require a more complex implementation in the packaging library, which would be a significant departure from the current approach.
Footnotes
Acknowledgements
Thanks to Filipe LaÃns for the suggestion of the abi_features attribute and
to Stephen Rosen for the Backwards Compatibility section of PEP 735, which
served as a template for the corresponding section in this PEP.
Change History
- 05-Nov-2026
- Rewrote Backwards Compatibility and clarified how installation and locking tools should reject package versions with unsupported markers and consider alternatives. Prohibited recovery that changes dependency semantics, while allowing static analysis to collect potential dependencies. Documented the limitations of fallback for source builds.
- Added the tool behavior appendix, documenting pip, uv, PDM, and Poetry behavior when encountering unknown markers.
- Introduced Core Metadata version 2.7 for
Requires-Distentries usingsys_abi_info, with producer, publishing tool, and index validation rules. Other dependency-specifier contexts do not require this version. - Specified how tools determine ABI features when
sys.abi_infoor its fields are unavailable, including false defaults forfree_threadedanddebugon non-CPython implementations. Defined how future missing attributes are handled. - Clarified that
free_threadeddescribes the interpreter’s build configuration rather than its runtime GIL state. - Clarified exact-value membership tests, behavior for absent values, and support for boolean, integer, and string values. Required tools to ignore fields with other value types.
- Added consistent dependency metadata across sdists and wheels as a motivation, referencing the PEP 825 investigation of 32-bit Windows. Related the compatibility guidance to PEP 825’s new environment markers.
- Updated the free-threading rollout discussion and the relationship to PEP 751, and added the independent uv prototype under Reference Implementation.
- Removed the Open Issues section, all open issues were addressed.
Appendices
Copyright
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.