Skip to content

Commit 845335a

Browse files
jrfnljrfnl
committed
PHP 8.0 | Add support for named function call arguments
PHP 8.0 introduces named function call parameters: ```php array_fill(start_index: 0, num: 100, value: 50); // Using reserved keywords as names is allowed. array_foobar(array: $array, switch: $switch, class: $class); ``` Ref: https://wiki.php.net/rfc/named_params This PR adds support to PHPCS for named function call arguments by adding a special custom token `T_PARAM_NAME` and tokenizing the _labels_ in function calls using named arguments to that new token, as per the proposal in 3159. I also ensured that the colon _after_ a parameter label is always tokenized as `T_COLON`. Includes some minor efficiency fixes to the code which deals with the colon vs inline else determination as there is no need to run the "is this a return type" or the "is this a `case` statement" checks if it has already been established that the colon is a colon and not an inline else. Includes a ridiculous amount of unit tests to safeguard the correct tokenization of both the parameter label as well as the colon after it (and potential inline else colons in the same statement). Please also see my comment about this here: #3159 (comment) **Note**: The only code samples I could come up with which would result in "incorrect" tokenization to `T_PARAM_NAME` are all either parse errors or compile errors. I've elected to let those tokenize as `T_PARAM_NAME` anyway as: 1. When there is a parse error/compile error, there will be more tokenizer issues anyway, so working around those cases seems redundant. 2. The code will at least tokenize consistently (the same) across PHP versions. (which wasn't the case for parse errors/compile errors with numeric literals or arrow functions, which is why they needed additional safeguards previously). Fixes 3159
1 parent cda358f commit 845335a

File tree

5 files changed

+1410
-45
lines changed

5 files changed

+1410
-45
lines changed

‎package.xml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,8 @@ http://pear.php.net/dtd/package-2.0.xsd">
157157
<file baseinstalldir="" name="BackfillNumericSeparatorTest.php" role="test" />
158158
<file baseinstalldir="" name="BitwiseOrTest.inc" role="test" />
159159
<file baseinstalldir="" name="BitwiseOrTest.php" role="test" />
160+
<file baseinstalldir="" name="NamedFunctionCallArgumentsTest.inc" role="test" />
161+
<file baseinstalldir="" name="NamedFunctionCallArgumentsTest.php" role="test" />
160162
<file baseinstalldir="" name="NullsafeObjectOperatorTest.inc" role="test" />
161163
<file baseinstalldir="" name="NullsafeObjectOperatorTest.php" role="test" />
162164
<file baseinstalldir="" name="ScopeSettingWithNamespaceOperatorTest.inc" role="test" />
@@ -2045,6 +2047,8 @@ http://pear.php.net/dtd/package-2.0.xsd">
20452047
<install as="CodeSniffer/Core/Tokenizer/BackfillNumericSeparatorTest.inc" name="tests/Core/Tokenizer/BackfillNumericSeparatorTest.inc" />
20462048
<install as="CodeSniffer/Core/Tokenizer/BitwiseOrTest.php" name="tests/Core/Tokenizer/BitwiseOrTest.php" />
20472049
<install as="CodeSniffer/Core/Tokenizer/BitwiseOrTest.inc" name="tests/Core/Tokenizer/BitwiseOrTest.inc" />
2050+
<install as="CodeSniffer/Core/Tokenizer/NamedFunctionCallArgumentsTest.php" name="tests/Core/Tokenizer/NamedFunctionCallArgumentsTest.php" />
2051+
<install as="CodeSniffer/Core/Tokenizer/NamedFunctionCallArgumentsTest.inc" name="tests/Core/Tokenizer/NamedFunctionCallArgumentsTest.inc" />
20482052
<install as="CodeSniffer/Core/Tokenizer/NullsafeObjectOperatorTest.php" name="tests/Core/Tokenizer/NullsafeObjectOperatorTest.php" />
20492053
<install as="CodeSniffer/Core/Tokenizer/NullsafeObjectOperatorTest.inc" name="tests/Core/Tokenizer/NullsafeObjectOperatorTest.inc" />
20502054
<install as="CodeSniffer/Core/Tokenizer/ScopeSettingWithNamespaceOperatorTest.php" name="tests/Core/Tokenizer/ScopeSettingWithNamespaceOperatorTest.php" />
@@ -2117,6 +2121,8 @@ http://pear.php.net/dtd/package-2.0.xsd">
21172121
<install as="CodeSniffer/Core/Tokenizer/BackfillNumericSeparatorTest.inc" name="tests/Core/Tokenizer/BackfillNumericSeparatorTest.inc" />
21182122
<install as="CodeSniffer/Core/Tokenizer/BitwiseOrTest.php" name="tests/Core/Tokenizer/BitwiseOrTest.php" />
21192123
<install as="CodeSniffer/Core/Tokenizer/BitwiseOrTest.inc" name="tests/Core/Tokenizer/BitwiseOrTest.inc" />
2124+
<install as="CodeSniffer/Core/Tokenizer/NamedFunctionCallArgumentsTest.php" name="tests/Core/Tokenizer/NamedFunctionCallArgumentsTest.php" />
2125+
<install as="CodeSniffer/Core/Tokenizer/NamedFunctionCallArgumentsTest.inc" name="tests/Core/Tokenizer/NamedFunctionCallArgumentsTest.inc" />
21202126
<install as="CodeSniffer/Core/Tokenizer/NullsafeObjectOperatorTest.php" name="tests/Core/Tokenizer/NullsafeObjectOperatorTest.php" />
21212127
<install as="CodeSniffer/Core/Tokenizer/NullsafeObjectOperatorTest.inc" name="tests/Core/Tokenizer/NullsafeObjectOperatorTest.inc" />
21222128
<install as="CodeSniffer/Core/Tokenizer/ScopeSettingWithNamespaceOperatorTest.php" name="tests/Core/Tokenizer/ScopeSettingWithNamespaceOperatorTest.php" />

‎src/Tokenizers/PHP.php

Lines changed: 123 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -893,6 +893,62 @@ protected function tokenize($string)
893893
continue;
894894
}//end if
895895

896+
/*
897+
Tokenize the parameter labels for PHP 8.0 named parameters as a special T_PARAM_NAME
898+
token and ensure that the colon after it is always T_COLON.
899+
*/
900+
901+
if ($tokenIsArray === true
902+
&& preg_match('`^[a-zA-Z_\x80-\xff]`', $token[1]) === 1
903+
) {
904+
// Get the next non-empty token.
905+
for ($i = ($stackPtr + 1); $i < $numTokens; $i++) {
906+
if (is_array($tokens[$i]) === false
907+
|| isset(Util\Tokens::$emptyTokens[$tokens[$i][0]]) === false
908+
) {
909+
break;
910+
}
911+
}
912+
913+
if (isset($tokens[$i]) === true
914+
&& is_array($tokens[$i]) === false
915+
&& $tokens[$i] === ':'
916+
) {
917+
// Get the previous non-empty token.
918+
for ($j = ($stackPtr - 1); $j > 0; $j--) {
919+
if (is_array($tokens[$j]) === false
920+
|| isset(Util\Tokens::$emptyTokens[$tokens[$j][0]]) === false
921+
) {
922+
break;
923+
}
924+
}
925+
926+
if (is_array($tokens[$j]) === false
927+
&& ($tokens[$j] === '('
928+
|| $tokens[$j] === ',')
929+
) {
930+
$newToken = [];
931+
$newToken['code'] = T_PARAM_NAME;
932+
$newToken['type'] = 'T_PARAM_NAME';
933+
$newToken['content'] = $token[1];
934+
$finalTokens[$newStackPtr] = $newToken;
935+
936+
$newStackPtr++;
937+
938+
// Modify the original token stack so that future checks, like
939+
// determining T_COLON vs T_INLINE_ELSE can handle this correctly.
940+
$tokens[$stackPtr][0] = T_PARAM_NAME;
941+
942+
if (PHP_CODESNIFFER_VERBOSITY > 1) {
943+
$type = Util\Tokens::tokenName($token[0]);
944+
echo "\t\t* token $stackPtr changed from $type to T_PARAM_NAME".PHP_EOL;
945+
}
946+
947+
continue;
948+
}
949+
}//end if
950+
}//end if
951+
896952
/*
897953
Before PHP 7.0, the "yield from" was tokenized as
898954
T_YIELD, T_WHITESPACE and T_STRING. So look for
@@ -1700,76 +1756,98 @@ function return types. We want to keep the parenthesis map clean,
17001756
// Convert colons that are actually the ELSE component of an
17011757
// inline IF statement.
17021758
if (empty($insideInlineIf) === false && $newToken['code'] === T_COLON) {
1703-
// Make sure this isn't a return type separator.
17041759
$isInlineIf = true;
1760+
1761+
// Make sure this isn't a named parameter label.
1762+
// Get the previous non-empty token.
17051763
for ($i = ($stackPtr - 1); $i > 0; $i--) {
17061764
if (is_array($tokens[$i]) === false
1707-
|| ($tokens[$i][0] !== T_DOC_COMMENT
1708-
&& $tokens[$i][0] !== T_COMMENT
1709-
&& $tokens[$i][0] !== T_WHITESPACE)
1765+
|| isset(Util\Tokens::$emptyTokens[$tokens[$i][0]]) === false
17101766
) {
17111767
break;
17121768
}
17131769
}
17141770

1715-
if ($tokens[$i] === ')') {
1716-
$parenCount = 1;
1717-
for ($i--; $i > 0; $i--) {
1718-
if ($tokens[$i] === '(') {
1719-
$parenCount--;
1720-
if ($parenCount === 0) {
1721-
break;
1722-
}
1723-
} else if ($tokens[$i] === ')') {
1724-
$parenCount++;
1725-
}
1771+
if ($tokens[$i][0] === T_PARAM_NAME) {
1772+
$isInlineIf = false;
1773+
if (PHP_CODESNIFFER_VERBOSITY > 1) {
1774+
echo "\t\t* token is parameter label, not T_INLINE_ELSE".PHP_EOL;
17261775
}
1776+
}
17271777

1728-
// We've found the open parenthesis, so if the previous
1729-
// non-empty token is FUNCTION or USE, this is a return type.
1730-
// Note that we need to skip T_STRING tokens here as these
1731-
// can be function names.
1732-
for ($i--; $i > 0; $i--) {
1778+
if ($isInlineIf === true) {
1779+
// Make sure this isn't a return type separator.
1780+
for ($i = ($stackPtr - 1); $i > 0; $i--) {
17331781
if (is_array($tokens[$i]) === false
17341782
|| ($tokens[$i][0] !== T_DOC_COMMENT
17351783
&& $tokens[$i][0] !== T_COMMENT
1736-
&& $tokens[$i][0] !== T_WHITESPACE
1737-
&& $tokens[$i][0] !== T_STRING)
1784+
&& $tokens[$i][0] !== T_WHITESPACE)
17381785
) {
17391786
break;
17401787
}
17411788
}
17421789

1743-
if ($tokens[$i][0] === T_FUNCTION || $tokens[$i][0] === T_FN || $tokens[$i][0] === T_USE) {
1744-
$isInlineIf = false;
1745-
if (PHP_CODESNIFFER_VERBOSITY > 1) {
1746-
echo "\t\t* token is return type, not T_INLINE_ELSE".PHP_EOL;
1790+
if ($tokens[$i] === ')') {
1791+
$parenCount = 1;
1792+
for ($i--; $i > 0; $i--) {
1793+
if ($tokens[$i] === '(') {
1794+
$parenCount--;
1795+
if ($parenCount === 0) {
1796+
break;
1797+
}
1798+
} else if ($tokens[$i] === ')') {
1799+
$parenCount++;
1800+
}
17471801
}
1748-
}
1802+
1803+
// We've found the open parenthesis, so if the previous
1804+
// non-empty token is FUNCTION or USE, this is a return type.
1805+
// Note that we need to skip T_STRING tokens here as these
1806+
// can be function names.
1807+
for ($i--; $i > 0; $i--) {
1808+
if (is_array($tokens[$i]) === false
1809+
|| ($tokens[$i][0] !== T_DOC_COMMENT
1810+
&& $tokens[$i][0] !== T_COMMENT
1811+
&& $tokens[$i][0] !== T_WHITESPACE
1812+
&& $tokens[$i][0] !== T_STRING)
1813+
) {
1814+
break;
1815+
}
1816+
}
1817+
1818+
if ($tokens[$i][0] === T_FUNCTION || $tokens[$i][0] === T_FN || $tokens[$i][0] === T_USE) {
1819+
$isInlineIf = false;
1820+
if (PHP_CODESNIFFER_VERBOSITY > 1) {
1821+
echo "\t\t* token is return type, not T_INLINE_ELSE".PHP_EOL;
1822+
}
1823+
}
1824+
}//end if
17491825
}//end if
17501826

17511827
// Check to see if this is a CASE or DEFAULT opener.
1752-
$inlineIfToken = $insideInlineIf[(count($insideInlineIf) - 1)];
1753-
for ($i = $stackPtr; $i > $inlineIfToken; $i--) {
1754-
if (is_array($tokens[$i]) === true
1755-
&& ($tokens[$i][0] === T_CASE
1756-
|| $tokens[$i][0] === T_DEFAULT)
1757-
) {
1758-
$isInlineIf = false;
1759-
if (PHP_CODESNIFFER_VERBOSITY > 1) {
1760-
echo "\t\t* token is T_CASE or T_DEFAULT opener, not T_INLINE_ELSE".PHP_EOL;
1761-
}
1828+
if ($isInlineIf === true) {
1829+
$inlineIfToken = $insideInlineIf[(count($insideInlineIf) - 1)];
1830+
for ($i = $stackPtr; $i > $inlineIfToken; $i--) {
1831+
if (is_array($tokens[$i]) === true
1832+
&& ($tokens[$i][0] === T_CASE
1833+
|| $tokens[$i][0] === T_DEFAULT)
1834+
) {
1835+
$isInlineIf = false;
1836+
if (PHP_CODESNIFFER_VERBOSITY > 1) {
1837+
echo "\t\t* token is T_CASE or T_DEFAULT opener, not T_INLINE_ELSE".PHP_EOL;
1838+
}
17621839

1763-
break;
1764-
}
1840+
break;
1841+
}
17651842

1766-
if (is_array($tokens[$i]) === false
1767-
&& ($tokens[$i] === ';'
1768-
|| $tokens[$i] === '{')
1769-
) {
1770-
break;
1843+
if (is_array($tokens[$i]) === false
1844+
&& ($tokens[$i] === ';'
1845+
|| $tokens[$i] === '{')
1846+
) {
1847+
break;
1848+
}
17711849
}
1772-
}
1850+
}//end if
17731851

17741852
if ($isInlineIf === true) {
17751853
array_pop($insideInlineIf);

‎src/Util/Tokens.php

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@
7676
define('T_ZSR_EQUAL', 'PHPCS_T_ZSR_EQUAL');
7777
define('T_FN_ARROW', 'T_FN_ARROW');
7878
define('T_TYPE_UNION', 'T_TYPE_UNION');
79+
define('T_PARAM_NAME', 'T_PARAM_NAME');
7980

8081
// Some PHP 5.5 tokens, replicated for lower versions.
8182
if (defined('T_FINALLY') === false) {

0 commit comments

Comments
 (0)