gh-148690: Build abi3t-compat\python3.dll on Windows by encukou · Pull Request #148912 · python/cpython

encukou · 2026-04-23T12:19:24Z

Per discussion in #146636 (comment), this adds abi3t-compat\python3.dll to Windows builds.

And to the WiX installer, though I haven't managed to test this.

Issue: Free-threaded and GIL-enabled Windows builds should share an abi3 libpython dll #148690

clin1234

Looks congruent to me

zooba · 2026-04-23T17:41:20Z

Looks like a solid change, but probably simpler to just create a python3t.vcxproj and put the python3t.dll in the default location, which will remove a lot of the modifications (and can simplify things you didn't even add, since we'll just build it twice and python3.dll and python3t.dll are now both constants). We rearrange output files as part of building the installer, so that's where the directory should come into it.

It won't work so nicely for dev/in-tree builds, but I'm not so concerned about that. We probably need to choose an entirely new output directory for free-threaded builds, but that's going to break all sorts of stuff that assumes directory names in a build, so probably best to just say "ABI3 will get weird in dev builds if you don't clean before switching kind of build". I expect anyone trying to work on both simultaneously has separate clones anyway.

chris-eibl · 2026-04-23T20:05:39Z

I've tried to build the installer using your branch and get the same error as CI:

error CNDL0027: The File/@Name attribute's value, 'abi3t-compat\python3t.dll', is not a valid long name because it contains illegal characters.

Changing to

            <Component Id="python_abi3tcompat.dll" Directory="abi3t_compat" Guid="*">
                <File Id="python_abi3tcompat.dll" Name="python$(var.MajorVersionNumber)t.dll" KeyPath="yes" />
            </Component>

and adding the directory to Tools/msi/common.wxs:

    <Fragment>
        <DirectoryRef Id="InstallDirectory">
            <Directory Id="abi3t_compat" Name="abi3t-compat" />
        </DirectoryRef>
    </Fragment>

Note Id="abi3t_compat" is chosen, because - is not allowed in an identifier.

As Steve mentions

put the python3t.dll in the default location,

then the above did work for me (when manually copying the dll just to try it out).

Otherwise, you'd have to add a Source in

<File Id="python_abi3tcompat.dll" Name="python$(var.MajorVersionNumber)t.dll" KeyPath="yes" />

But this is all new to me as well ...

chris-eibl · 2026-04-23T20:57:22Z

Adding

        <LinkerBindInputPaths Include="$(BuildPath)">
            <BindName>build</BindName>
        </LinkerBindInputPaths>

to Tools\msi\msi.props and using

<File Id="python_abi3tcompat.dll" Name="python$(var.MajorVersionNumber)t.dll" Source="!(bindpath.build)\abi3t-compat\python$(var.MajorVersionNumber)t.dll" KeyPath="yes" />

did work for me using your current layout.

chris-eibl · 2026-04-23T20:59:20Z

And IMHO we should bundle python3t_d.dll in the installer as well.

zooba · 2026-04-24T09:16:15Z

Changing to

            <Component Id="python_abi3tcompat.dll" Directory="abi3t_compat" Guid="*">
                <File Id="python_abi3tcompat.dll" Name="python$(var.MajorVersionNumber)t.dll" KeyPath="yes" />
            </Component>

and adding the directory to Tools/msi/common.wxs:

    <Fragment>
        <DirectoryRef Id="InstallDirectory">
            <Directory Id="abi3t_compat" Name="abi3t-compat" />
        </DirectoryRef>
    </Fragment>

This is the way.

this is all new to me as well ...

In two more weeks we get to never touch this MSI again :)

encukou · 2026-04-28T15:58:59Z

Thanks for the pointers!

This is how far I got in another day: python3dll and python3tdll with fixed names.
Further pointers would be appreciated again :)
I got it to build on CI, and on my machine once, but now Tools\msi\build.bat -x64 is giving me:

Build FAILED.

"C:\Users\encukou\dev\cpython\Tools\msi\bundle\snapshot.wixproj" (default target) (1) ->
"C:\Users\encukou\dev\cpython\Tools\msi\doc\doc.wixproj" (Build target) (7) ->
(Link target) ->
  C:\Users\encukou\dev\cpython\Tools\msi\doc\doc.wxs(12): error LGHT0094: Unresolved reference to symbol 'WixComponen 
tGroup:doc_html' in section 'Product:*'. [C:\Users\encukou\dev\cpython\Tools\msi\doc\doc.wixproj]
  C:\Users\encukou\dev\cpython\Tools\msi\doc\doc.wxs(27): error LGHT0094: Unresolved reference to symbol 'WixComponen 
tGroup:doc_html' in section 'Product:*'. [C:\Users\encukou\dev\cpython\Tools\msi\doc\doc.wixproj]

I'll continue tomorrow.

zooba · 2026-04-28T18:07:51Z

I think you need to build docs manually for the installer? I remember cutting a corner in the msi/build.bat that I regretted, but solved it by basically never using that script again... buildrelease.bat should do it all.

Couple of other comments/concerns coming on the changes.

ngoldbaum · 2026-04-28T18:46:22Z

I built an installer based on this branch, but am seeing issues caused by the python3t.dll produced being self-referential:

C:\Users\goldbaum\AppData\Local\Programs\Python\Python315>dumpbin /exports abi3t-compat\python3t.dll
Microsoft (R) COFF/PE Dumper Version 14.44.35226.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file abi3t-compat\python3t.dll

File Type: DLL

  Section contains the following exports for python3t.dll

    00000000 characteristics
    FFFFFFFF time date stamp
        0.00 version
           1 ordinal base
         948 number of functions
         948 number of names

    ordinal hint RVA      name

          1    0          PyABIInfo_Check (forwarded to python3t.PyABIInfo_Check)
          2    1          PyAIter_Check (forwarded to python3t.PyAIter_Check)
          3    2          PyArg_Parse (forwarded to python3t.PyArg_Parse)
          4    3          PyArg_ParseTuple (forwarded to python3t.PyArg_ParseTuple)
          5    4          PyArg_ParseTupleAndKeywords (forwarded to python3t.PyArg_ParseTupleAndKeywords)

In this output we should be seeing e.g. PyABIInfo_Check re-exporting from python315.dll, but it's re-exporting from itself.

ngoldbaum · 2026-04-28T18:56:49Z

I think you need to build docs manually for the installer?

Yeah, all the CI jobs pass --doc, so building without it seems to have broken at some point.

I have a (claude-suggested) fix for the issue I described in my last comment that I'm testing out. Sadly each build takes about a half hour end-to-end, including the Maturin and PyO3 builds needed in the stable-abi-testing repo to run end-to-end tests.

zooba

The other place that will need changing is the script under PC/layout, which is the "new" install creator (essentially it's the make install command). That just has to copy in the right file if it's been built, without the subdirectory.

zooba · 2026-04-28T19:34:45Z

+            }
+#endif
+
            wcscpy(p + 1, PY3_DLLNAME);


We also want to search for the compatibility DLL name without the extra directory, but only if the one with the directory doesn't exist. That will be the long-term (post 3.15) path, and also used by the PyManager package straight away (which doesn't do overlapping installs).

zooba · 2026-04-28T20:11:12Z

caused by the python3t.dll produced being self-referential

This will be a typo in the script that generates the .def file. I missed it when reviewing, but I assume it's just a wrong variable that'll be easy to spot (for Petr, who has worked on this generator a lot).

ngoldbaum

With the suggested changes I'm able to run the tests in the stable-abi-testing repo on the GIL-enabled and free-threaded builds and don't see any issues. There are two failures on the GIL-enabled build for the meson-python tests, but fixing those will require updating @mgorny's meson-python branch we use in the stable-abi-testing repo or adjusting the tests in the stable-abi-testing repo.

I also manually verified using my maturin and PyO3 branches for abi3t support that a wheel built with either build is installable and importable by both builds.

Not sure how well these suggestions play with @zooba's suggestions to leave things unconditional. Hopefully there's a straightforward way to integrate my suggestions with his.

ngoldbaum · 2026-04-28T19:58:58Z

    <Fragment>
        <ComponentGroup Id="core_dll">
+            <Component Id="python_abi3tcompat.dll" Directory="abi3t_compat" Guid="*">
+                <File Id="python_abi3tcompat.dll" Name="python$(var.MajorVersionNumber)t.dll" Source="python$(var.MajorVersionNumber)t.dll" KeyPath="yes" />


Suggested change

<File Id="python_abi3tcompat.dll" Name="python$(var.MajorVersionNumber)t.dll" Source="python$(var.MajorVersionNumber)t.dll" KeyPath="yes" />

<File Id="python_abi3tcompat.dll" Name="python$(var.MajorVersionNumber)t.dll" Source="abi3t-compat\python$(var.MajorVersionNumber)t.dll" KeyPath="yes" />

encukou

Thanks! I'll start today with these suggestions.

Co-authored-by: Steve Dower <steve.dower@microsoft.com> Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>

read-the-docs-community · 2026-04-29T06:09:21Z

Documentation build overview

📚 cpython-previews | 🛠️ Build #32462471 | 📁 Comparing 3dc7bc1 against main (d71e3bc)

🔍 Preview build

6 files changed · ± 6 modified

± Modified

zooba · 2026-04-29T14:48:06Z

I looked at the error message in the installer build and immediately thought "that'll be @hugovk's fault" and one git log Doc\make.bat later I see I was right 😆

Maybe we shouldn't be assuming that every dependency (pip, in this case) is forcibly updated on every build? That's actually bad policy (you should use a lockfile to pin to a known good version, which might be a little older 😉 )

hugovk · 2026-04-29T15:13:39Z

It looks like currently we're using whatever pip is in the CI runner?

Short term, we can either require newer pip or revert the make.bat change, so docs' deps are installed from requirements.txt instead of pylock.toml.

And/or we can start pinning pip to a certain version. We should then decide when/how to upgrade that. Easiest would be via Dependabot.

Which do you prefer?

ngoldbaum · 2026-04-29T16:10:50Z

I tried re-doing my testing from yesterday but hit a segfault. See this crash report from WinDbg using the stable-abi-testing maturin/rust example:

https://gist.github.com/ngoldbaum/7f277b39f8c054959c47a96e4ec5c228

Or this crash using the setuptools/c example:

https://gist.github.com/ngoldbaum/fa31157e5839c8425bdd6626d0d8ef3d

It's possible that this is caused by other changes that were recently merged into CPython main that somehow broke my WIP branch for PyO3 PEP 803 support. Let me try again with the branch I got to successfully work yesterday and see what's different in that version of things.

ngoldbaum · 2026-04-29T16:52:09Z

I think I see the issue - the free-threaded python3t.dll is clobbering the python3t.dll that gets produced in the first build pass for the GIL-enabled build, so the abi3t_compat\python3t.dll that ends up installed re-exports from the wrong dll, and we get crashes because sizeof(PyObject) is wrong:

C:\Users\goldbaum\AppData\Local\Programs\Python\Python315>dumpbin /exports abi3t-compat\python3t.dll
Microsoft (R) COFF/PE Dumper Version 14.44.35226.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file abi3t-compat\python3t.dll

File Type: DLL

  Section contains the following exports for python3t.dll

    00000000 characteristics
    FFFFFFFF time date stamp
        0.00 version
           1 ordinal base
         948 number of functions
         948 number of names

    ordinal hint RVA      name

          1    0          PyABIInfo_Check (forwarded to python315t.PyABIInfo_Check)
          2    1          PyAIter_Check (forwarded to python315t.PyAIter_Check)
          3    2          PyArg_Parse (forwarded to python315t.PyArg_Parse)
          4    3          PyArg_ParseTuple (forwarded to python315t.PyArg_ParseTuple)
          5    4          PyArg_ParseTupleAndKeywords (forwarded to python315t.PyArg_ParseTupleAndKeywords)

All the forwards above should be from python315, not python315t.

encukou added 4 commits April 23, 2026 12:21

Link against python3t.lib

5db5a47

Add ABI3T_COMPAT_DLLNAME

1153d48

Build abi3t-compat\python3t.dll

fda3900

Add abi3tcompat to MSI installer

96676cf

encukou requested review from a team, FFY00 and ericsnowcurrently as code owners April 23, 2026 12:19

bedevere-app Bot added the awaiting core review label Apr 23, 2026

bedevere-app Bot mentioned this pull request Apr 23, 2026

Free-threaded and GIL-enabled Windows builds should share an abi3 libpython dll #148690

Open

encukou added the skip news label Apr 23, 2026

clin1234 reviewed Apr 23, 2026

View reviewed changes

ngoldbaum and others added 4 commits April 28, 2026 11:35

minimal fix for build errors on Petr's PR

32083dd

Merge in the main branch

e25d7ff

Build python3.dll & python3t.dll separately, with fixed names

eb61469

Don't use !(bindpath.build)

92ae703

zooba reviewed Apr 28, 2026

View reviewed changes

ngoldbaum reviewed Apr 28, 2026

View reviewed changes

encukou commented Apr 29, 2026

View reviewed changes

Comment thread PCbuild/_remote_debugging.vcxproj Outdated

Apply suggestions from code review

3dc7bc1

Co-authored-by: Steve Dower <steve.dower@microsoft.com> Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>

encukou added 3 commits April 29, 2026 12:17

Look for python3t.dll without the directory too

b64ad2c

Merge in the main branch

9dd8cae

PC/layout: Include python3t in GIL-ful builds

4bc0289

	<File Id="python_abi3tcompat.dll" Name="python$(var.MajorVersionNumber)t.dll" Source="python$(var.MajorVersionNumber)t.dll" KeyPath="yes" />
	<File Id="python_abi3tcompat.dll" Name="python$(var.MajorVersionNumber)t.dll" Source="abi3t-compat\python$(var.MajorVersionNumber)t.dll" KeyPath="yes" />

Uh oh!

Conversation

encukou commented Apr 23, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clin1234 left a comment

Choose a reason for hiding this comment

Uh oh!

zooba commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chris-eibl commented Apr 23, 2026

Uh oh!

chris-eibl commented Apr 23, 2026

Uh oh!

chris-eibl commented Apr 23, 2026

Uh oh!

zooba commented Apr 24, 2026

Uh oh!

encukou commented Apr 28, 2026

Uh oh!

zooba commented Apr 28, 2026

Uh oh!

ngoldbaum commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ngoldbaum commented Apr 28, 2026

Uh oh!

zooba left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zooba Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

zooba commented Apr 28, 2026

Uh oh!

ngoldbaum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ngoldbaum Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

encukou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

read-the-docs-community Bot commented Apr 29, 2026

Documentation build overview

Uh oh!

zooba commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hugovk commented Apr 29, 2026

Uh oh!

ngoldbaum commented Apr 29, 2026

Uh oh!

ngoldbaum commented Apr 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

encukou commented Apr 23, 2026 •

edited by bedevere-app Bot

Loading

zooba commented Apr 23, 2026 •

edited

Loading

ngoldbaum commented Apr 28, 2026 •

edited

Loading

zooba commented Apr 29, 2026 •

edited

Loading