Skip to content

[clang-format] Recognize TableGen paste operator on separate line #133722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sstwcw merged 3 commits into from
Apr 10, 2025
Merged

[clang-format] Recognize TableGen paste operator on separate line #133722

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b018450
[clang-format] Recognize TableGen paste operator on separate line
sstwcw Mar 31, 2025
d3671a5
Add other directives
sstwcw Apr 4, 2025
45c25b7
Comments
sstwcw Apr 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions clang/lib/Format/TokenAnnotator.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -949,6 +949,8 @@ class AnnotatingParser {
HashTok->setType(TT_Unknown);
if (!parseTableGenValue(ParseNameMode))
return false;
if (!CurrentToken)
return true;
}
// In name mode, '{' is regarded as the end of the value.
// See TGParser::ParseValue in TGParser.cpp
Expand Down
11 changes: 9 additions & 2 deletions clang/lib/Format/UnwrappedLineParser.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4853,9 +4853,16 @@ void UnwrappedLineParser::readToken(int LevelDifference) {
PreviousWasComment = FormatTok->is(tok::comment);

while (!Line->InPPDirective && FormatTok->is(tok::hash) &&
(!Style.isVerilog() ||
Keywords.isVerilogPPDirective(*Tokens->peekNextToken())) &&
FirstNonCommentOnLine) {
// In Verilog, the backtick is used for macro invocations. In TableGen,
// the single hash is used for the paste operator.
const FormatToken *Next = Tokens->peekNextToken();
assert(Next); // There is an EOF token at the end.
Copy link
Contributor

@owenca owenca Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const FormatToken *Next = Tokens->peekNextToken();
assert(Next); // There is an EOF token at the end.
const auto *Next = Tokens->peekNextToken();

It seems that peekNextToken() never returns null.

Copy link
Contributor Author

@sstwcw sstwcw Apr 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that asserts were for things that never happen. So I added one. Is this something that should happen even less so that asserts are no good?

Copy link
Contributor

@owenca owenca Apr 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO assert() should not be used on something that's obviously true, and for this reason we never assert on peekNextToken().

if ((Style.isVerilog() && !Keywords.isVerilogPPDirective(*Next)) ||
(Style.isTableGen() &&
!Next->isOneOf(tok::pp_define, tok::pp_ifdef, tok::pp_ifndef))) {
break;
}
distributeComments(Comments, FormatTok);
Comments.clear();
// If there is an unfinished unwrapped line, we flush the preprocessor
Expand Down
6 changes: 6 additions & 0 deletions clang/unittests/Format/FormatTestTableGen.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,12 @@ TEST_F(FormatTestTableGen, PasteOperator) {
" string Z = [\"Traring\", \"Paste\", \"Traring\", \"Paste\",\n"
" \"Traring\", \"Paste\"]#;\n"
"}");
verifyFormat("def x#x {}", "def x\n"
"#x {}");
verifyFormat("def x#x {}", "def x\n"
"#\n"
"x {}");
verifyFormat("def x#x");
}

TEST_F(FormatTestTableGen, ClassDefinition) {
Expand Down
17 changes: 17 additions & 0 deletions clang/unittests/Format/TokenAnnotatorTest.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2832,6 +2832,23 @@ TEST_F(TokenAnnotatorTest, UnderstandTableGenTokens) {
Tokens = Annotate("!cond");
EXPECT_TOKEN(Tokens[0], tok::identifier, TT_TableGenCondOperator);

// The paste operator should not be treated as a preprocessor directive even
// if it is on a separate line.
Tokens = Annotate("def x\n"
"#embed {}");
ASSERT_EQ(Tokens.size(), 7u) << Tokens;
EXPECT_TOKEN(Tokens[1], tok::identifier, TT_StartOfName);
EXPECT_TOKEN(Tokens[2], tok::hash, TT_Unknown);
EXPECT_EQ(Tokens[1]->Next, Tokens[2]);
Tokens = Annotate("def x\n"
"#define x\n"
"#embed {}");
ASSERT_EQ(Tokens.size(), 10u) << Tokens;
EXPECT_TOKEN(Tokens[1], tok::identifier, TT_StartOfName);
EXPECT_TOKEN(Tokens[2], tok::hash, TT_Unknown);
EXPECT_TOKEN(Tokens[5], tok::hash, TT_Unknown);
EXPECT_EQ(Tokens[1]->Next, Tokens[5]);

auto AnnotateValue = [this, &Style](StringRef Code) {
// Values are annotated only in specific context.
auto Result = annotate(("def X { let V = " + Code + "; }").str(), Style);
Expand Down