On Windows and OS X, this simply adds a bunch of extra builds. On Linux,
I'm also bumping the manylinux tag because Python 3.11 is newer than the
distro in question. This means that there will be no builds for "new
Python on old distros" because one has to draw a line somewhere.
Effectively, wheels for Linux Python 3.11 will therefore require glibc
2.35.
I don't have the details about why exactly that version of pybind11 on
Ubuntu 20.04 broke down, but here's how it ended up on Ubuntu 22.04 when
trying to use Python 3.11:
In file included from /usr/include/pybind11/cast.h:16,
from /usr/include/pybind11/attr.h:13,
from /usr/include/pybind11/pybind11.h:13,
from /usr/include/pybind11/functional.h:12,
from /home/runner/work/oopt-gnpy-libyang/oopt-gnpy-libyang/oopt-gnpy-libyang.cpp:1:
/usr/include/pybind11/detail/type_caster_base.h: In function ‘std::string pybind11::detail::error_string()’:
/usr/include/pybind11/detail/type_caster_base.h:482:26: error: invalid use of incomplete type ‘PyFrameObject’ {aka ‘struct _frame’}
482 | frame = frame->f_back;
| ^~
In file included from /opt/hostedtoolcache/Python/3.11.1/x64/include/python3.11/Python.h:42,
from /usr/include/pybind11/detail/common.h:215,
from /usr/include/pybind11/pytypes.h:12,
from /usr/include/pybind11/cast.h:13,
from /usr/include/pybind11/attr.h:13,
from /usr/include/pybind11/pybind11.h:13,
from /usr/include/pybind11/functional.h:12,
from /home/runner/work/oopt-gnpy-libyang/oopt-gnpy-libyang/oopt-gnpy-libyang.cpp:1:
/opt/hostedtoolcache/Python/3.11.1/x64/include/python3.11/pytypedefs.h:22:16: note: forward declaration of ‘PyFrameObject’ {aka ‘struct _frame’}
22 | typedef struct _frame PyFrameObject;
| ^~~~~~
On Windows, it failed because (apparently) vcpkg patches Python
detection as per https://github.com/microsoft/vcpkg/issues/23796. The
build would fail like this (vcpkg Pytohn is 3.10, the system-selected
one is 3.11):
CMake Error at C:/t/build-env-1a5f954n/Lib/site-packages/cmake/data/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find Python (missing: Development.Module) (found suitable version
"3.11.1", minimum required is "3.8")
Call Stack (most recent call first):
C:/t/build-env-1a5f954n/Lib/site-packages/cmake/data/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
C:/t/build-env-1a5f954n/Lib/site-packages/cmake/data/share/cmake-3.25/Modules/FindPython/Support.cmake:3245 (find_package_handle_standard_args)
C:/t/build-env-1a5f954n/Lib/site-packages/cmake/data/share/cmake-3.25/Modules/FindPython.cmake:519 (include)
So let's just use the PIP magic and hope for the best.
TL;DR: the path limit strikes again
The build infrastructure produces excessively long path names, and our
wonderful combination of pypa-build -> cmake -> MSVC chokes on them on a
spectacular fashion even though the runner image already has that
long-path-name support set in the registry:
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe - broken
CMake Error at C:/Users/runneradmin/AppData/Local/Temp/build-env-b5qtu_yx/Lib/site-packages/cmake/data/share/cmake-3.25/Modules/CMakeTestCXXCompiler.cmake:63 (message):
The C++ compiler
"C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: C:/Users/runneradmin/AppData/Local/Temp/build-via-sdist-nzuhfzad/oopt-gnpy-libyang-0.0.5.dev5+g470406b/build/temp.win-amd64-cpython-310/Release/oopt_gnpy_libyang/CMakeFiles/CMakeScratch/TryCompile-5bq90s
Run Build Command(s):C:/Program Files/Microsoft Visual Studio/2022/Enterprise/MSBuild/Current/Bin/amd64/MSBuild.exe cmTC_1c273.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /v:m && MSBuild version 17.4.1+9a89d02ff for .NET Framework
-- Configuring incomplete, errors occurred!
Microsoft (R) C/C++ Optimizing Compiler Version 19.34.31937 for x64
See also "C:/Users/runneradmin/AppData/Local/Temp/build-via-sdist-nzuhfzad/oopt-gnpy-libyang-0.0.5.dev5+g470406b/build/temp.win-amd64-cpython-310/Release/oopt_gnpy_libyang/CMakeFiles/CMakeOutput.log".
Copyright (C) Microsoft Corporation. All rights reserved.
See also "C:/Users/runneradmin/AppData/Local/Temp/build-via-sdist-nzuhfzad/oopt-gnpy-libyang-0.0.5.dev5+g470406b/build/temp.win-amd64-cpython-310/Release/oopt_gnpy_libyang/CMakeFiles/CMakeError.log".
testCXXCompiler.cxx
cl /c /Zi /W3 /WX- /diagnostics:column /Od /Ob0 /D _MBCS /D WIN32 /D _WINDOWS /D "CMAKE_INTDIR=\"Debug\"" /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /GR /Fo"cmTC_1c273.dir\Debug\\" /Fd"cmTC_1c273.dir\Debug\vc143.pdb" /external:W3 /Gd /TP /wd4251 /wd4275 /errorReport:queue "C:\Users\runneradmin\AppData\Local\Temp\build-via-sdist-nzuhfzad\oopt-gnpy-libyang-0.0.5.dev5+g470406b\build\temp.win-amd64-cpython-310\Release\oopt_gnpy_libyang\CMakeFiles\CMakeScratch\TryCompile-5bq90s\testCXXCompiler.cxx"
FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\runneradmin\AppData\Local\Temp\build-via-sdist-nzuhfzad\oopt-gnpy-libyang-0.0.5.dev5+g470406b\build\temp.win-amd64-cpython-310\Release\oopt_gnpy_libyang\CMakeFiles\CMakeScratch\TryCompile-5bq90s\cmTC_1c273.dir\Debug\cmTC_1c273.tlog\link-rc.read.1.tlog. The system cannot find the path specified. [C:\Users\runneradmin\AppData\Local\Temp\build-via-sdist-nzuhfzad\oopt-gnpy-libyang-0.0.5.dev5+g470406b\build\temp.win-amd64-cpython-310\Release\oopt_gnpy_libyang\CMakeFiles\CMakeScratch\TryCompile-5bq90s\cmTC_1c273.vcxproj]
Just look at the beauty of the generated path. Let me break it down into
easily digestible chunks. First of all, this is the global temporary
directory as set by the GitHub environment:
C:\Users\runneradmin\AppData\Local\Temp\
40 characters, thank you. Let's add some unique temp thingy for the build:
build-via-sdist-nzuhfzad\
25 characters, 16 of which are useless. OK.
oopt-gnpy-libyang-0.0.5.dev5+g470406b\
38 characters for the package name and version. We have a long package name.
build\temp.win-amd64-cpython-310\Release\
41 useless characters indicating that this is a build (as if we didn't
know from path part #2). Let's also stick in some random info about
platform type, Python type and its version because nothing can happen,
right?
oopt_gnpy_libyang\
18 extra characters in case the project name changed compared from path
chunk #3.
CMakeFiles\CMakeScratch\TryCompile-5bq90s\cmTC_1c273.dir\Debug\
CMake to the rescue, 63 characters for guaranteeing that this is a
frakin' unique path.
cmTC_1c273.tlog\link-rc.read.1.tlog
At this point MSVC adds meek 35 characters which once again are not a
big deal, right?
Apple clang is too randomly broken (no operator<=> for std::string,
seriously?), so this needs GCC.
Also, there were some non-fatal warnings about a mismatch of "deployment
target", so I figured out that I probably need to use the latest and
greatest to limit the blast size when stuff breaks. Fingers crossed;
these blind builds really take a leap of faith. But the resulting
CPython module loads successfully, so I suppose this might actually
work?
Since Apple clang is not enough and the bundled GCC is not a
cross-compiler (and I don't really feel like bootstrapping one today),
we cannot build arm64 binaries on Mac OS yet. But the Python version
that is driving this build is the `universal2` fat binary thingy, and
due to pypa/wheel#406 and/or pypa/setuptools#2520, this required another
fair amount of hoop jumping. Finally, the BSD-ish userland comes with
its set of peculiarities.
While that one has a bigger chance of producing binaries which work
on older systems, it has doctest 2.3.6, whereas upstream libyang-cpp
needs doctest 2.4.6.
Since we have full control over the whole build stack (do we, when it
comes to Python?), it's safe to disable MSVC warnings about exporting
symbols from a DLL which are inherited from a non-exported type (or have
members which are of a non-exported type).
