|
2 | 2 |
|
3 | 3 | This file contains important instructions for working on the phpdoc-parser project. |
4 | 4 |
|
| 5 | +## Project Overview |
| 6 | + |
| 7 | +**phpstan/phpdoc-parser** is a library that represents PHPDocs with an Abstract Syntax Tree (AST). It supports parsing and modifying PHPDocs, and is primarily used by PHPStan for static analysis. |
| 8 | + |
| 9 | +### Key Features |
| 10 | +- Parses PHPDoc comments into an AST representation |
| 11 | +- Supports all PHPDoc tags and types (see [PHPStan documentation](https://phpstan.org/writing-php-code/phpdocs-basics)) |
| 12 | +- Format-preserving printer for modifying and printing AST nodes |
| 13 | +- Support for Doctrine Annotations parsing |
| 14 | +- Nullable, intersection, generic, and conditional types support |
| 15 | + |
| 16 | +### Requirements |
| 17 | +- PHP ^7.4 || ^8.0 |
| 18 | +- Platform target: PHP 7.4.6 |
| 19 | + |
| 20 | +## Project Structure |
| 21 | + |
| 22 | +### Source Code (`src/`) |
| 23 | + |
| 24 | +The source code is organized into the following main components: |
| 25 | + |
| 26 | +1. **Lexer** (`src/Lexer/`) |
| 27 | + - `Lexer.php` - Tokenizes PHPDoc strings |
| 28 | + |
| 29 | +2. **Parser** (`src/Parser/`) |
| 30 | + - `PhpDocParser.php` - Main PHPDoc parser (parses tags and structure) |
| 31 | + - `TypeParser.php` - Parses PHPDoc type expressions |
| 32 | + - `ConstExprParser.php` - Parses constant expressions |
| 33 | + - `TokenIterator.php` - Iterator for tokens |
| 34 | + - `StringUnescaper.php` - Handles string unescaping |
| 35 | + - `ParserException.php` - Exception handling |
| 36 | + |
| 37 | +3. **AST** (`src/Ast/`) |
| 38 | + - `Node.php` - Base AST node interface |
| 39 | + - `NodeTraverser.php` - Traverses and transforms AST |
| 40 | + - `NodeVisitor.php` - Visitor pattern for AST traversal |
| 41 | + - `Type/` - Type nodes (GenericTypeNode, ArrayTypeNode, UnionTypeNode, etc.) |
| 42 | + - `PhpDoc/` - PHPDoc tag nodes (ParamTagValueNode, ReturnTagValueNode, etc.) |
| 43 | + - `ConstExpr/` - Constant expression nodes |
| 44 | + - `NodeVisitor/` - Built-in visitors (CloningVisitor, etc.) |
| 45 | + |
| 46 | +4. **Printer** (`src/Printer/`) |
| 47 | + - `Printer.php` - Prints AST back to PHPDoc format |
| 48 | + - `Differ.php` - Computes differences between AST nodes |
| 49 | + - `DiffElem.php` - Represents diff elements |
| 50 | + |
| 51 | +5. **Configuration** |
| 52 | + - `ParserConfig.php` - Parser configuration (attributes to use) |
| 53 | + |
| 54 | +### Tests (`tests/PHPStan/`) |
| 55 | + |
| 56 | +Tests mirror the source structure and include: |
| 57 | + |
| 58 | +1. **Parser Tests** (`tests/PHPStan/Parser/`) |
| 59 | + - `TypeParserTest.php` - Type parsing tests |
| 60 | + - `PhpDocParserTest.php` - PHPDoc parsing tests |
| 61 | + - `ConstExprParserTest.php` - Constant expression parsing tests |
| 62 | + - `FuzzyTest.php` - Fuzzy testing |
| 63 | + - `Doctrine/` - Doctrine annotation test fixtures |
| 64 | + |
| 65 | +2. **AST Tests** (`tests/PHPStan/Ast/`) |
| 66 | + - `NodeTraverserTest.php` - Node traversal tests |
| 67 | + - `Attributes/AttributesTest.php` - AST attribute tests |
| 68 | + - `ToString/` - Tests for converting AST to string |
| 69 | + - `NodeVisitor/` - Visitor pattern tests |
| 70 | + |
| 71 | +3. **Printer Tests** (`tests/PHPStan/Printer/`) |
| 72 | + - Tests for format-preserving printing functionality |
| 73 | + |
| 74 | +### Configuration Files |
| 75 | + |
| 76 | +- `phpunit.xml` - PHPUnit test configuration |
| 77 | +- `phpstan.neon` - PHPStan static analysis configuration |
| 78 | +- `phpstan-baseline.neon` - PHPStan baseline (known issues) |
| 79 | +- `phpcs.xml` - PHP CodeSniffer configuration |
| 80 | +- `composer.json` - Dependencies and autoloading |
| 81 | + |
| 82 | +## How the Parser Works |
| 83 | + |
| 84 | +The parsing flow follows these steps: |
| 85 | + |
| 86 | +1. **Lexing**: `Lexer` tokenizes the PHPDoc string into tokens |
| 87 | +2. **Parsing**: `PhpDocParser` uses `TypeParser` and `ConstExprParser` to build an AST |
| 88 | +3. **Traversal/Modification**: `NodeTraverser` with `NodeVisitor` can traverse and modify the AST |
| 89 | +4. **Printing**: `Printer` converts the AST back to PHPDoc format (optionally preserving formatting) |
| 90 | + |
| 91 | +### Basic Usage Example |
| 92 | + |
| 93 | +```php |
| 94 | +$config = new ParserConfig(usedAttributes: []); |
| 95 | +$lexer = new Lexer($config); |
| 96 | +$constExprParser = new ConstExprParser($config); |
| 97 | +$typeParser = new TypeParser($config, $constExprParser); |
| 98 | +$phpDocParser = new PhpDocParser($config, $typeParser, $constExprParser); |
| 99 | + |
| 100 | +$tokens = new TokenIterator($lexer->tokenize('/** @param Lorem $a */')); |
| 101 | +$phpDocNode = $phpDocParser->parse($tokens); |
| 102 | +``` |
| 103 | + |
| 104 | +### Format-Preserving Printing |
| 105 | + |
| 106 | +For format-preserving printing (used when modifying existing PHPDocs), enable these attributes: |
| 107 | +- `lines` - Preserve line information |
| 108 | +- `indexes` - Preserve token indexes |
| 109 | +- `comments` - Preserve comments |
| 110 | + |
| 111 | +## Common Development Tasks |
| 112 | + |
| 113 | +### Adding a New PHPDoc Tag |
| 114 | +1. Create a new `*TagValueNode` class in `src/Ast/PhpDoc/` |
| 115 | +2. Add parsing logic in `PhpDocParser.php` |
| 116 | +3. Add tests in `tests/PHPStan/Parser/PhpDocParserTest.php` |
| 117 | +4. Run tests and PHPStan |
| 118 | + |
| 119 | +### Adding a New Type Node |
| 120 | +1. Create a new `*TypeNode` class in `src/Ast/Type/` |
| 121 | +2. Add parsing logic in `TypeParser.php` |
| 122 | +3. Add printing logic in `Printer.php` |
| 123 | +4. Add tests in `tests/PHPStan/Parser/TypeParserTest.php` |
| 124 | +5. Run tests and PHPStan |
| 125 | + |
| 126 | +### Modifying the Lexer |
| 127 | +1. Update token generation in `Lexer.php` |
| 128 | +2. Update parsers that consume those tokens |
| 129 | +3. Add/update tests |
| 130 | +4. Run comprehensive checks with `make check` |
| 131 | + |
5 | 132 | ## Testing and Quality Checks |
6 | 133 |
|
7 | 134 | ### Running Tests |
@@ -82,3 +209,58 @@ make check |
82 | 209 | 6. Repeat as needed |
83 | 210 |
|
84 | 211 | **Remember: Tests and PHPStan MUST pass before any commit.** |
| 212 | + |
| 213 | +## Coding Standards and Best Practices |
| 214 | + |
| 215 | +### Code Style |
| 216 | +- Follow PSR-12 coding standards (enforced by phpcs) |
| 217 | +- Use tabs for indentation (project convention) |
| 218 | +- Run `make cs-fix` to automatically fix code style issues |
| 219 | +- Always run `make cs` to verify code style before committing |
| 220 | + |
| 221 | +### PHPStan Rules |
| 222 | +- Project uses strict PHPStan rules (level max) |
| 223 | +- All code must pass static analysis |
| 224 | +- Avoid adding to phpstan-baseline.neon unless absolutely necessary |
| 225 | +- Type hints are required for all public APIs |
| 226 | + |
| 227 | +### Testing Best Practices |
| 228 | +- All new features must include tests |
| 229 | +- Tests should be in the corresponding test directory matching src/ structure |
| 230 | +- Use data providers for testing multiple similar cases |
| 231 | +- Test both valid and invalid inputs |
| 232 | +- Include edge cases and error conditions |
| 233 | + |
| 234 | +### AST Node Conventions |
| 235 | +- All AST nodes implement the `Node` interface |
| 236 | +- Nodes should be immutable where possible |
| 237 | +- Use `__toString()` for debugging output |
| 238 | +- Implement proper equality checks if needed |
| 239 | +- Follow the visitor pattern for AST traversal |
| 240 | + |
| 241 | +### Parser Patterns |
| 242 | +- Parsers should be recursive descent style |
| 243 | +- Use `TokenIterator` for token consumption |
| 244 | +- Throw `ParserException` for syntax errors |
| 245 | +- Support optional attributes through `ParserConfig` |
| 246 | +- Maintain backwards compatibility when adding features |
| 247 | + |
| 248 | +## Important Notes |
| 249 | + |
| 250 | +### Backwards Compatibility |
| 251 | +- This library is used by PHPStan and many other tools |
| 252 | +- Breaking changes should be avoided |
| 253 | +- New features should be opt-in when possible |
| 254 | +- Deprecate before removing functionality |
| 255 | + |
| 256 | +### Performance Considerations |
| 257 | +- The parser is performance-critical (runs on large codebases) |
| 258 | +- Avoid unnecessary object allocations |
| 259 | +- Be careful with regex patterns |
| 260 | +- Consider memory usage in loops |
| 261 | + |
| 262 | +### Documentation |
| 263 | +- Public APIs should have PHPDoc comments |
| 264 | +- Complex logic should include inline comments |
| 265 | +- Update README.md when adding major features |
| 266 | +- Reference PHPStan documentation for PHPDoc tag specifications |
0 commit comments