Skip to content

Commit 3c2fc7a

Browse files
authored
[flang] Refine tokenization trick that hid macro name (#121990)
In order to properly expose the Hollerith editing item in something like FORMAT(3I9HHOLLERITH) as its own token, the tokenization routine in the prescanner has special handling for digit strings followed by letters ("3I" above). This handler's effects are too broad, and prevent a macro name from being recognized as such in a reported bug; make the test for a hidden Hollerith more precise. Fixes #121931.
1 parent 3a8a52f commit 3c2fc7a

File tree

2 files changed

+21
-3
lines changed

2 files changed

+21
-3
lines changed

flang/lib/Parser/prescan.cpp

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -709,9 +709,22 @@ bool Prescanner::NextToken(TokenSequence &tokens) {
709709
QuotedCharacterLiteral(tokens, start);
710710
} else if (IsLetter(*at_) && !preventHollerith_ &&
711711
parenthesisNesting_ > 0) {
712-
// Handles FORMAT(3I9HHOLLERITH) by skipping over the first I so that
713-
// we don't misrecognize I9HOLLERITH as an identifier in the next case.
714-
EmitCharAndAdvance(tokens, *at_);
712+
const char *p{at_};
713+
int digits{0};
714+
for (;; ++digits) {
715+
++p;
716+
if (InFixedFormSource()) {
717+
p = SkipWhiteSpace(p);
718+
}
719+
if (!IsDecimalDigit(*p)) {
720+
break;
721+
}
722+
}
723+
if (digits > 0 && (*p == 'h' || *p == 'H')) {
724+
// Handles FORMAT(3I9HHOLLERITH) by skipping over the first I so that
725+
// we don't misrecognize I9HOLLERITH as an identifier in the next case.
726+
EmitCharAndAdvance(tokens, *at_);
727+
}
715728
}
716729
preventHollerith_ = false;
717730
} else if (*at_ == '.') {

flang/test/Preprocessing/bug129131.F

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
! RUN: %flang -fc1 -fdebug-unparse %s 2>&1 | FileCheck %s
2+
! CHECK: PRINT *, 2_4
3+
#define a ,3
4+
print *, mod(5 a)
5+
end

0 commit comments

Comments
 (0)