[lldb] Require at least Python 3.11 when targeting the Limited C API

This is a continuation of my effort [1] to eliminate uses of the private
Python API in LLDB. I found 3 more issues that were hidden in the
typemap, because those were included before we defined Py_LIMITED_API.

The primary motivator for bumping the minimum required Python version is
our use of `pybuffer` (i.e. the Buffer Protocol).

 - For Python versions prior to Python 3.11, your only option was to
   target the "Old Buffer Protocol" [2], which has been deprecated since
   Python 3 and removed in Python 3.13, though the symbols still exist
   for ABI compatibility.

 - For Python versions 3.13 and later, because of the removal, your only
   option is to target the "New Buffer Protocol" [3] which was only
   added to the stable API in Python 3.11.

This leads to certain combination of Python versions for building
against and targeting with the limited API that are not compatible. For
example, you cannot target the 3.8 stable API when building against
Python 3.13 because the new version of Python doesn't declare the old
APIs and you cannot use the new APIs yet.

To make this more complicated, the use of the buffer protocol is coming
from SWIG. Furthermore, all released versions of SWIG get this wrong.
For unrelated reasons, to work around a bug in SWIG < 4.1, we already
reimplement the swig type wrapper for the Buffer Protocol in a way
that's compatible with Python 3.11 and later. Instead of waiting for a
SWIG release that fixes this issue (it's already fixed on the master
branch) we can keep the workaround and bump the minimum Python version
when targeting the limited C API.

[1] #151617
[2] https://docs.python.org/3.12/c-api/objbuffer.html
[3] https://docs.python.org/3/c-api/buffer.html

Author: JDevlieghere

lldb/bindings/python/python-typemaps.swig

@@ -220,19 +220,19 @@ AND call SWIG_fail at the same time, because it will result in a double free.
 }
 
 // Disable default type checking for this method to avoid SWIG dispatch issues.
-// 
+//
 // Problem: SBThread::GetStopDescription has two overloads:
-//   1. GetStopDescription(char* dst_or_null, size_t dst_len) 
+//   1. GetStopDescription(char* dst_or_null, size_t dst_len)
 //   2. GetStopDescription(lldb::SBStream& stream)
 //
 // SWIG generates a dispatch function to select the correct overload based on argument types.
 // see https://www.swig.org/Doc4.0/SWIGDocumentation.html#Typemaps_overloading.
 // However, this dispatcher doesn't consider typemaps that transform function signatures.
 //
 // In Python, our typemap converts GetStopDescription(char*, size_t) to GetStopDescription(int).
-// The dispatcher still checks against the original (char*, size_t) signature instead of 
+// The dispatcher still checks against the original (char*, size_t) signature instead of
 // the transformed (int) signature, causing type matching to fail.
-// This only affects SBThread::GetStopDescription since the type check also matches 
+// This only affects SBThread::GetStopDescription since the type check also matches
 // the argument name, which is unique to this function.
 %typemap(typecheck, precedence=SWIG_TYPECHECK_POINTER) (char *dst_or_null, size_t dst_len) ""
 
@@ -628,12 +628,11 @@ template <> bool SetNumberFromPyObject<double>(double &number, PyObject *obj) {
   }
 }
 
-// These two pybuffer macros are copied out of swig/Lib/python/pybuffer.i,
-// and fixed so they will not crash if PyObject_GetBuffer fails.
-// https://github.com/swig/swig/issues/1640
-//
-// I've also moved the call to PyBuffer_Release to the end of the SWIG wrapper,
-// doing it right away is not legal according to the python buffer protocol.
+// These two pybuffer macros are copied out of swig/Lib/python/pybuffer.i.
+// - Avoids crash if PyObject_GetBuffer fails and delays calling
+//   PyBuffer_Release (https://github.com/swig/swig/issues/1640)
+// - Avoids using deprecated "old buffer protocol" APIs when targeting the
+//   Stable C API.
 %inline %{
 struct Py_buffer_RAII {
   Py_buffer buffer = {};

lldb/bindings/python/python-wrapper.swig

@@ -128,8 +128,11 @@ bool lldb_private::python::SWIGBridge::LLDBSwigPythonCallTypeScript(
 
   PyObject *pfunc_impl = nullptr;
 
-  if (pyfunct_wrapper && *pyfunct_wrapper &&
-      PyFunction_Check(*pyfunct_wrapper)) {
+  if (pyfunct_wrapper && *pyfunct_wrapper 
+#ifndef Py_LIMITED_API
+      && PyFunction_Check(*pyfunct_wrapper)
+#endif
+      ) {
     pfunc_impl = (PyObject *)(*pyfunct_wrapper);
     if (pfunc_impl->ob_refcnt == 1) {
       Py_XDECREF(pfunc_impl);

lldb/bindings/python/python.swig

@@ -59,6 +59,11 @@ except ImportError:
 // Parameter types will be used in the autodoc string.
 %feature("autodoc", "1");
 
+// Include lldb-python first as it sets Py_LIMITED_API.
+%begin %{
+#include "../source/Plugins/ScriptInterpreter/Python/lldb-python.h"
+%}
+
 %pythoncode%{
 import uuid
 import re

lldb/cmake/modules/LLDBConfig.cmake

@@ -180,12 +180,14 @@ if (LLDB_ENABLE_PYTHON)
       "Path to use as PYTHONHOME in lldb. If a relative path is specified, it will be resolved at runtime relative to liblldb directory.")
   endif()
 
-  if (SWIG_VERSION VERSION_GREATER_EQUAL "4.2" AND NOT LLDB_EMBED_PYTHON_HOME)
+  if (SWIG_VERSION VERSION_GREATER_EQUAL "4.2"
+      Python3_VERSION VERSION_GREATER_EQUAL "3.11"
+      AND NOT LLDB_EMBED_PYTHON_HOME)
     set(default_enable_python_limited_api ON)
   else()
     set(default_enable_python_limited_api OFF)
   endif()
-  option(LLDB_ENABLE_PYTHON_LIMITED_API "Force LLDB to only use the Python Limited API (requires SWIG 4.2 or later)"
+  option(LLDB_ENABLE_PYTHON_LIMITED_API "Only use the Python Limited API (requires at least Python 3.11 and SWIG 4.2)"
     ${default_enable_python_limited_api})
 else()
   # Even if Python scripting is disabled, we still need a Python interpreter to

lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h

@@ -45,12 +45,13 @@ static llvm::Expected<bool> *g_fcxx_modules_workaround [[maybe_unused]];
 #include <locale>
 #endif
 
-#define LLDB_MINIMUM_PYTHON_VERSION 0x03080000
-
-#if LLDB_ENABLE_PYTHON_LIMITED_API
 // If defined, LLDB will be ABI-compatible with all Python 3 releases from the
 // specified one onward, and can use Limited API introduced up to that version.
+#if LLDB_ENABLE_PYTHON_LIMITED_API
+#define LLDB_MINIMUM_PYTHON_VERSION 0x030b0000
 #define Py_LIMITED_API LLDB_MINIMUM_PYTHON_VERSION
+#else
+#define LLDB_MINIMUM_PYTHON_VERSION 0x03080000
 #endif
 
 // Include python for non windows machines