More docs updates

Author: nikic

doc/component/AST_builders.markdown

@@ -101,7 +101,7 @@ abstract class SomeOtherClass extends SomeClass implements A\Few, \Interfaces
      *
      * @param SomeClass And takes a parameter
      */
-    public abstract function someMethod(SomeClass $someParam) : bool;
+    public abstract function someMethod(SomeClass $someParam): bool;
     protected function anotherMethod($someParam = 'test')
     {
         print $someParam;

doc/component/Constant_expression_evaluation.markdown

@@ -19,9 +19,9 @@ PHP-Parser supports evaluation of such constant expressions through the `ConstEx
 
 use PhpParser\{ConstExprEvaluator, ConstExprEvaluationException};
 
-$evalutator = new ConstExprEvaluator();
+$evaluator = new ConstExprEvaluator();
 try {
-    $value = $evalutator->evaluateSilently($someExpr);
+    $value = $evaluator->evaluateSilently($someExpr);
 } catch (ConstExprEvaluationException $e) {
     // Either the expression contains unsupported expression types,
     // or an error occurred during evaluation
@@ -69,6 +69,7 @@ expressions, apart from the following:
  * `Scalar\MagicConst\*`
  * `Expr\ConstFetch` (only null/false/true are handled)
  * `Expr\ClassConstFetch`
+ * `Expr\New_` (since PHP 8.1)
 
 Handling these expression types requires non-local information, such as which global constants are
 defined. By default, the evaluator will throw a `ConstExprEvaluationException` when it encounters

doc/component/Error_handling.markdown

@@ -14,10 +14,10 @@ In order to receive information about not only the line, but also the column spa
 position attributes in the lexer need to be enabled:
 
 ```php
-$lexer = new PhpParser\Lexer(array(
+$lexerOptions = array(
     'usedAttributes' => array('comments', 'startLine', 'endLine', 'startFilePos', 'endFilePos'),
-));
-$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, $lexer);
+);
+$parser = (new PhpParser\ParserFactory())->createForHostVersion($lexerOptions);
 
 try {
     $stmts = $parser->parse($code);
@@ -56,7 +56,7 @@ To instead collect all encountered errors into an array, while trying to continu
 an instance of `ErrorHandler\Collecting` can be passed to the `Parser::parse()` method. A usage example:
 
 ```php
-$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::ONLY_PHP7);
+$parser = (new PhpParser\ParserFactory())->createForHostVersion();
 $errorHandler = new PhpParser\ErrorHandler\Collecting;
 
 $stmts = $parser->parse($code, $errorHandler);
@@ -72,4 +72,6 @@ if (null !== $stmts) {
 }
 ```
 
+The partial AST may contain `Expr\Error` nodes that indicate that an error occurred while parsing an expression.
+
 The `NameResolver` visitor also accepts an `ErrorHandler` as a constructor argument.

doc/component/FAQ.markdown

@@ -19,7 +19,7 @@ $code = '...';
 $traverser = new NodeTraverser;
 $traverser->addVisitor(new ParentConnectingVisitor);
 
-$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
+$parser = (new ParserFactory())->createForHostVersion();
 $ast    = $parser->parse($code);
 $ast    = $traverser->traverse($ast);
 ```
@@ -42,7 +42,7 @@ $code = '...';
 $traverser = new NodeTraverser;
 $traverser->addVisitor(new NodeConnectingVisitor);
 
-$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
+$parser = (new ParserFactory())->createForHostVersion();
 $ast    = $parser->parse($code);
 $ast    = $traverser->traverse($ast);
 ```

doc/component/JSON_representation.markdown

@@ -18,7 +18,7 @@ function printLine($msg) {
 }
 CODE;
 
-$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
+$parser = (new ParserFactory())->createForHostVersion();
 
 try {
     $stmts = $parser->parse($code);

doc/component/Lexer.markdown

@@ -42,16 +42,17 @@ The attributes used in this example match the default behavior of the lexer. The
 > **Note:** The example in this section is outdated in that this information is directly available in the AST: While
 > `$property->isPublic()` does not distinguish between `public` and `var`, directly checking `$property->flags` for
 > the `$property->flags & Class_::VISIBILITY_MODIFIER_MASK) === 0` allows making this distinction without resorting to
-> tokens. However the general idea behind the example still applies in other cases.
+> tokens. However, the general idea behind the example still applies in other cases.
 
 The token offset information is useful if you wish to examine the exact formatting used for a node. For example the AST
 does not distinguish whether a property was declared using `public` or using `var`, but you can retrieve this
 information based on the token position:
 
 ```php
+/** @param PhpParser\Token[] $tokens */
 function isDeclaredUsingVar(array $tokens, PhpParser\Node\Stmt\Property $prop) {
-    $i = $prop->getAttribute('startTokenPos');
-    return $tokens[$i][0] === T_VAR;
+    $i = $prop->getStartTokenPos();
+    return $tokens[$i]->id === T_VAR;
 }
 ```
 
@@ -72,12 +73,12 @@ class MyNodeVisitor extends PhpParser\NodeVisitorAbstract {
     }
 }
 
-$lexer = new PhpParser\Lexer(array(
+$lexerOptions = array(
     'usedAttributes' => array(
         'comments', 'startLine', 'endLine', 'startTokenPos', 'endTokenPos'
     )
-));
-$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::ONLY_PHP7, $lexer);
+);
+$parser = (new PhpParser\ParserFactory())->createForHostVersion($lexerOptions);
 
 $visitor = new MyNodeVisitor();
 $traverser = new PhpParser\NodeTraverser();
@@ -111,14 +112,15 @@ The `startLexing()` method is invoked whenever the `parse()` method of the parse
 code that is to be lexed (including the opening tag). It can be used to reset state or preprocess the source code or tokens. The
 passed `ErrorHandler` should be used to report lexing errors.
 
-The `getTokens()` method returns the current token array, in the usual `token_get_all()` format. This method is not
-used by the parser (which uses `getNextToken()`), but is useful in combination with the token position attributes.
+The `getTokens()` method returns the current array of `PhpParser\Token`s, which are compatible with the PHP 8 `PhpToken`
+class. This method is not used by the parser (which uses `getNextToken()`), but is useful in combination with the token
+position attributes.
 
 The `handleHaltCompiler()` method is called whenever a `T_HALT_COMPILER` token is encountered. It has to return the
 remaining string after the construct (not including `();`).
 
-The `getNextToken()` method returns the ID of the next token (as defined by the `Parser::T_*` constants). If no more
-tokens are available it must return `0`, which is the ID of the `EOF` token. Furthermore the string content of the
+The `getNextToken()` method returns the ID of the next token (in the sense of `Token::$id`). If no more
+tokens are available it must return `0`, which is the ID of the `EOF` token. Furthermore, the string content of the
 token should be written into the by-reference `$value` parameter (which will then be available as `$n` in the parser).
 
 ### Attribute handling
@@ -144,10 +146,10 @@ class KeepOriginalValueLexer extends Lexer // or Lexer\Emulative
     public function getNextToken(&$value = null, &$startAttributes = null, &$endAttributes = null) {
         $tokenId = parent::getNextToken($value, $startAttributes, $endAttributes);
 
-        if ($tokenId == Tokens::T_CONSTANT_ENCAPSED_STRING   // non-interpolated string
-            || $tokenId == Tokens::T_ENCAPSED_AND_WHITESPACE // interpolated string
-            || $tokenId == Tokens::T_LNUMBER                 // integer
-            || $tokenId == Tokens::T_DNUMBER                 // floating point number
+        if ($tokenId == \T_CONSTANT_ENCAPSED_STRING   // non-interpolated string
+            || $tokenId == \T_ENCAPSED_AND_WHITESPACE // interpolated string
+            || $tokenId == \T_LNUMBER                 // integer
+            || $tokenId == \T_DNUMBER                 // floating point number
         ) {
             // could also use $startAttributes, doesn't really matter here
             $endAttributes['originalValue'] = $value;

doc/component/Name_resolution.markdown

@@ -10,7 +10,7 @@ visitor (NameResolver) based on it.
 The NameResolver visitor
 ------------------------
 
-The `NameResolver` visitor can (and for nearly all uses of the AST, is) be applied to resolve names
+The `NameResolver` visitor can (and for nearly all uses of the AST, should) be applied to resolve names
 to their fully-qualified form, to the degree that this is possible.
 
 ```php
@@ -53,7 +53,7 @@ name. Once again, if an unqualified function or constant name cannot be resolved
 `resolvedName` attribute will not be present, and instead a `namespacedName` attribute is added.
 
 The `replaceNodes` attribute is useful if you wish to perform modifications on the AST, as you
-probably do not wish the resoluting code to have fully resolved names as a side-effect.
+probably do not wish the resulting code to have fully resolved names as a side-effect.
 
 The NameContext
 ---------------
@@ -84,4 +84,4 @@ representation of a name given the current name resolution environment.
 The name context is intended to be used for name resolution operations outside the AST itself, such
 as class names inside doc comments. A visitor running in parallel with the name resolver can access
 the name context using `$nameResolver->getNameContext()`. Alternatively a visitor can use an
-independent context and explicitly feed `Namespace` and `Use` nodes to it.
+independent context and explicitly feed `Namespace` and `Use` nodes to it.

doc/component/Pretty_printing.markdown

@@ -31,10 +31,16 @@ expression.
 Customizing the formatting
 --------------------------
 
-Apart from an `shortArraySyntax` option, the default pretty printer does not provide any
-functionality to customize the formatting of the generated code. The pretty printer does respect a
-number of `kind` attributes used by some notes (e.g., whether an integer should be printed as
-decimal, hexadecimal, etc), but there are no options to control brace placement or similar.
+The pretty printer respects a number of `kind` attributes used by some notes (e.g., whether an
+integer should be printed as decimal, hexadecimal, etc). Additionally, it supports two options:
+
+* `phpVersion` (defaults to 7.0) allows opting into formatting that is not supported by older PHP
+  versions.
+* `shortArraySyntax` determines the used array syntax if the `kind` attribute is not set. This is
+  a legacy option, and `phpVersion` should be used to control this behavior instead.
+
+However, the default pretty printer does not provide any functionality for fine-grained
+customization of code formatting.
 
 If you want to make minor changes to the formatting, the easiest way is to extend the pretty printer
 and override the methods responsible for the node types you are interested in.
@@ -46,29 +52,27 @@ default pretty printer with an existing library for code reformatting, such as
 Formatting-preserving pretty printing
 -------------------------------------
 
-> **Note:** This functionality is **experimental** and not yet complete.
-
 For automated code refactoring, migration and similar, you will usually only want to modify a small
 portion of the code and leave the remainder alone. The basic pretty printer is not suitable for
 this, because it will also reformat parts of the code which have not been modified.
 
-Since PHP-Parser 4.0, an experimental formatting-preserving pretty-printing mode is available, which
+Since PHP-Parser 4.0, a formatting-preserving pretty-printing mode is available, which
 attempts to preserve the formatting of code (those AST nodes that have not changed) and only reformat
 code which has been modified or newly inserted.
 
 Use of the formatting-preservation functionality requires some additional preparatory steps:
 
 ```php
-use PhpParser\{Lexer, NodeTraverser, NodeVisitor, Parser, PrettyPrinter};
+use PhpParser\{Lexer, NodeTraverser, NodeVisitor, ParserFactory, PrettyPrinter};
 
-$lexer = new Lexer\Emulative([
+$lexerOptions = new [
     'usedAttributes' => [
         'comments',
         'startLine', 'endLine',
         'startTokenPos', 'endTokenPos',
     ],
-]);
-$parser = new Parser\Php7($lexer);
+];
+$parser = (new ParserFactory())->createForHostVersion($lexerOptions);
 
 $traverser = new NodeTraverser();
 $traverser->addVisitor(new NodeVisitor\CloningVisitor());
@@ -86,11 +90,9 @@ $newCode = $printer->printFormatPreserving($newStmts, $oldStmts, $oldTokens);
 ```
 
 If you make use of the name resolution functionality, you will likely want to disable the
-`replaceNodes` option. This will add resolved names as attributes, instead of directlying modifying
+`replaceNodes` option. This will add resolved names as attributes, instead of directly modifying
 the AST and causing spurious changes to the pretty printed code. For more information, see the
 [name resolution documentation](Name_resolution.markdown).
 
-This functionality is experimental and not yet fully implemented. It should not provide incorrect
-code, but it may sometimes reformat more code than necessary. Open issues are tracked in
-[issue #344](https://github.com/nikic/PHP-Parser/issues/344). If you encounter problems while using
-this functionality, please open an issue, so we know what to prioritize.
+The formatting-preservation works on a best-effort basis and may sometimes reformat more code tha
+necessary. If you encounter problems while using this functionality, please open an issue.

doc/component/Walking_the_AST.markdown

@@ -129,8 +129,7 @@ Now `$a && $b` will be replaced by `!($a && $b)`. Then the traverser will go int
 only) child of `!($a && $b)`, which is `$a && $b`. The transformation applies again and we end up
 with `!!($a && $b)`. This will continue until PHP hits the memory limit.
 
-Finally, two special replacement types are supported only by leaveNode. The first is removal of a
-node:
+Finally, there are two special replacement types. The first is removal of a node:
 
 ```php
 public function leaveNode(Node $node) {
@@ -165,8 +164,8 @@ This example will remove all calls to `var_dump()` which occur as expression sta
 that `var_dump($a);` will be removed, but `if (var_dump($a))` will not be removed (and there is no
 obvious way in which it can be removed).
 
-Next to removing nodes, it is also possible to replace one node with multiple nodes. Again, this
-only works inside leaveNode and only if the parent structure is an array.
+Next to removing nodes, it is also possible to replace one node with multiple nodes. This
+only works if the parent structure is an array.
 
 ```php
 public function leaveNode(Node $node) {