GH-47583: [CI][Python] Enable PyArrow Debug build on Windows CI job

[raulcd]

TBD

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

This PR includes breaking changes to public APIs. (If there are any breaking changes to public APIs, please explain which changes are breaking. If not, you can remove this.)

This PR contains a "Critical Fix". (If the changes fix either (a) a security vulnerability, (b) a bug that caused incorrect or invalid data to be produced, or (c) a bug that causes a crash (even when the API contract is upheld), please provide explanation. If not, you can remove this.)

• GitHub Issue: [CI][Python] Use Debug build on our Windows Python CI job #47583

[github-actions]

⚠️ GitHub issue #47583 has been automatically assigned in GitHub to PR creator.

[raulcd]

@pitrou the problem seems to be that Debug builds try to link python313_d.lib instead of python313.lib. This seems to be defined upstream on Python itself here (link for 3.16 but 3.13 is the same):
https://github.com/python/cpython/blob/2cd5b79284dd2331cbd9e11afabbfbf7e906103d/PC/pyconfig.h#L340-L342
And it fails with:

 LINK : fatal error LNK1104: cannot open file 'python313_d.lib'

On the Windows CI job we use actions/setup-python which do not allow to install CPython debug. I could try RelWithDebInfo instead of DEBUGotherwise I think we require installing CPython on Debug mode (manually?) which would not allow us to use the setup-python action.
Do you have other ideas?

[pitrou]

Ouch, I had missed that _DEBUG implied Py_DEBUG. This is annoying: we would like our own assertions (and the C++ stdlib's) without needing a special build of CPython :(

[pitrou]

See https://discuss.python.org/t/building-debug-version-of-extensions-on-windows-without-debug-binaries/105012

[pitrou]

According to the discussion above, perhaps we should use RelWithDebInfo (or even plain Release) and undefine the C preprocessor macro NDEBUG.

[pitrou]

I think we require installing CPython on Debug mode (manually?) which would not allow us to use the setup-python action

We can of course do this, assuming we find a readily available debug build of CPython on Windows, but then some dependencies might need to be recompiled from source and we'd rather avoid that.