[libclang/python] Return None instead of null cursors from Token.cursor #163183
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
llvmbot
added
clang
|
@llvm/pr-subscribers-clang Author: Jannick Kremer (DeinAlptraum) ChangesSince #138103 , the Fixes #163180 Full diff: https://github.com/llvm/llvm-project/pull/163183.diff 3 Files Affected:
diff --git a/clang/bindings/python/clang/cindex.py b/clang/bindings/python/clang/cindex.py
index 80140d2787608..da97bd375608c 100644
--- a/clang/bindings/python/clang/cindex.py
+++ b/clang/bindings/python/clang/cindex.py
@@ -3939,6 +3939,8 @@ def cursor(self):
cursor._tu = self._tu
conf.lib.clang_annotateTokens(self._tu, byref(self), 1, byref(cursor))
+ if cursor.is_null():
+ return None
return cursor
diff --git a/clang/bindings/python/tests/cindex/test_tokens.py b/clang/bindings/python/tests/cindex/test_tokens.py
index b6c1fc8b83600..d1982164deca8 100644
--- a/clang/bindings/python/tests/cindex/test_tokens.py
+++ b/clang/bindings/python/tests/cindex/test_tokens.py
@@ -53,3 +53,11 @@ def test_token_extent(self):
self.assertEqual(extent.start.offset, 4)
self.assertEqual(extent.end.offset, 7)
+
+ def test_null_cursor(self):
+ """Ensure that null cursors are converted to None by get_tokens"""
+ tu = get_tu("int i = 5;")
+ tokens = list(tu.get_tokens(extent=tu.cursor.extent))
+ for token in tokens:
+ print(token.spelling, token.kind, token.extent)
+ self.assertEqual(tokens[-1].cursor, None)
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 65b086caf3652..b32009309c1cb 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -136,6 +136,7 @@ Clang Frontend Potentially Breaking Changes
Clang Python Bindings Potentially Breaking Changes
--------------------------------------------------
+- Return ``None`` instead of null cursors from ``Token.cursor``
- TypeKind ``ELABORATED`` is not used anymore, per clang AST changes removing
ElaboratedTypes. The value becomes unused, and all the existing users should
expect the former underlying type to be reported instead.
|
Endilll
approved these changes
Oct 13, 2025
|
I talked to Aaron about backporting, and this should be fine to backport. |
|
Okay, so I get it into the 21.x branch by calling the cherry-pick command in a comment here? |
akadutta
pushed a commit
to akadutta/llvm-project
that referenced
this pull request
Oct 14, 2025
…or (llvm#163183) Since llvm#138103 , the `Cursor` class throws an error when any of its methods is called on a null cursor. Simultaneously, we adapted all methods to return `None` instead of a null cursor, so users should not encounter these. We have overlooked one way to end up with null cursors, namely the `Token.cursor` property, which may return null cursors under some circumstances. Fixes llvm#163180
|
/cherry-pick 38a5282 |
Error: Command failed due to missing milestone. |
|
/cherry-pick 38a5282 |
|
Failed to cherry-pick: 38a5282 https://github.com/llvm/llvm-project/actions/runs/18593736643 Please manually backport the fix and push it to your github fork. Once this is done, please create a pull request |
DeinAlptraum
added a commit
to DeinAlptraum/llvm-project
that referenced
this pull request
Oct 17, 2025
…or (llvm#163183) Since llvm#138103 , the `Cursor` class throws an error when any of its methods is called on a null cursor. Simultaneously, we adapted all methods to return `None` instead of a null cursor, so users should not encounter these. We have overlooked one way to end up with null cursors, namely the `Token.cursor` property, which may return null cursors under some circumstances. Fixes llvm#163180
|
Fixed the release note conflcit manually and opened a PR at #163961 |
c-rhodes
pushed a commit
to DeinAlptraum/llvm-project
that referenced
this pull request
Oct 20, 2025
…or (llvm#163183) Since llvm#138103 , the `Cursor` class throws an error when any of its methods is called on a null cursor. Simultaneously, we adapted all methods to return `None` instead of a null cursor, so users should not encounter these. We have overlooked one way to end up with null cursors, namely the `Token.cursor` property, which may return null cursors under some circumstances. Fixes llvm#163180
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since #138103 , the
Cursorclass throws an error when any of its methods is called on a null cursor. Simultaneously, we adapted all methods to returnNoneinstead of a null cursor, so users should not encounter these. We have overlooked one way to end up with null cursors, namely theToken.cursorproperty, which may return null cursors under some circumstances.Fixes #163180