diff --git a/src/docs/contribute/formatter.md b/src/docs/contribute/formatter.md index b3df766cdc..2ac72310e1 100644 --- a/src/docs/contribute/formatter.md +++ b/src/docs/contribute/formatter.md @@ -5,4 +5,37 @@ outline: deep # Formatter (Prettier) -We are currently porting prettier to Oxc. +We are currently porting Prettier and Biome Formatter to Oxc to create a high-performance, Prettier-compatible formatter. + +## Architecture Overview + +The Oxc formatter is built around the same core concepts as Prettier but with significant performance optimizations: + +- **Document Model**: Uses Prettier and Biome's document IR (Intermediate Representation) +- **Pretty Printing**: Implements Wadler's pretty printing algorithm +- **AST Integration**: Leverages Oxc's fast parser for optimal performance + +## Performance Considerations + +### Optimization Strategies + +- **Memory Arena**: AST allocated in bump allocator +- **String Interning**: Reuse common strings +- **Lazy Evaluation**: Defer expensive computations + +## Current Challenges + +### Technical Challenges + +1. **Comment Handling**: Preserving comment placement and formatting +2. **JavaScript Quirks**: Handling edge cases in JavaScript syntax +3. **Performance vs Compatibility**: Balancing speed with exact Prettier output +4. **Memory Management**: Efficient handling of large files + +### Missing Features + +- [ ] Plugin system compatibility +- [ ] Configuration file support +- [ ] Editor integrations +- [ ] CLI tool +- [ ] Language server protocol diff --git a/src/docs/contribute/introduction.md b/src/docs/contribute/introduction.md index eee6b44534..aea3ef6e59 100644 --- a/src/docs/contribute/introduction.md +++ b/src/docs/contribute/introduction.md @@ -3,10 +3,91 @@ title: Introduction outline: deep --- -# Introduction +# Contributing to Oxc -Thank you for your interest in contributing to Oxc! +Thank you for your interest in contributing to Oxc! We're building the next generation of JavaScript tooling, and we'd love your help. -Please check out our [good first issues](https://github.com/oxc-project/oxc/contribute) or ask for guidance on [Discord](https://discord.gg/9uXCAwqQZW). +## Quick Start -We welcome and appreciate any form of contributions. +The fastest way to get started is through our [good first issues](https://github.com/oxc-project/oxc/contribute). These are carefully selected tasks that are perfect for new contributors. + +Need guidance? Join our [Discord community](https://discord.gg/9uXCAwqQZW) where our team and community members are happy to help. + +## Ways to Contribute + +We welcome and appreciate any form of contributions: + +### ๐Ÿ› Bug Reports + +- Report parsing errors or incorrect linting behavior +- Share performance issues or regressions +- Document edge cases we haven't considered + +### ๐Ÿš€ Feature Development + +- Add new linting rules +- Improve parser conformance +- Enhance transformer capabilities +- Build new tools in the Oxc ecosystem + +### ๐Ÿ“š Documentation + +- Improve getting started guides +- Add examples and tutorials +- Document architecture decisions +- Translate content to other languages + +### ๐Ÿงช Testing + +- Add test cases from real-world codebases +- Improve test coverage +- Create performance benchmarks +- Test against ecosystem projects + +### ๐Ÿ”ง Infrastructure + +- Improve build and CI systems +- Enhance development tooling +- Optimize performance critical paths +- Maintain compatibility with other tools + +### Understanding the Codebase + +Oxc is organized into several crates: + +- **`oxc_parser`**: High-performance JavaScript/TypeScript parser +- **`oxc_linter`**: Fast linting engine with 500+ rules +- **`oxc_transformer`**: TypeScript and JSX transformation +- **`oxc_minifier`**: JavaScript minification (in development) +- **`oxc_formatter`**: Code formatting (in development) + +### Your First Contribution + +1. **Browse Issues**: Look for issues labeled [`good first issue`](https://github.com/oxc-project/oxc/labels/good%20first%20issue) +2. **Ask Questions**: Don't hesitate to ask for clarification on Discord or GitHub +3. **Start Small**: Begin with documentation improvements or small bug fixes +4. **Learn the Patterns**: Study existing code to understand our conventions + +## Community + +### Communication Channels + +- **GitHub Discussions**: For design discussions and questions +- **Discord**: Real-time chat with the team and community +- **GitHub Issues**: Bug reports and feature requests +- **Twitter**: Follow [@boshen_c](https://twitter.com/boshen_c) for updates + +### Code of Conduct + +We are committed to providing a welcoming and inclusive experience for everyone. Please read our [Code of Conduct](https://github.com/oxc-project/oxc/blob/main/CODE_OF_CONDUCT.md) before participating. + +## Next Steps + +Ready to contribute? Here are some great places to start: + +- ๐Ÿ“– **Learn More**: Check out our [development guide](./development.md) +- ๐Ÿ” **Find an Issue**: Browse our [good first issues](https://github.com/oxc-project/oxc/contribute) +- ๐Ÿ’ฌ **Join the Community**: Connect with us on [Discord](https://discord.gg/9uXCAwqQZW) +- ๐Ÿ› ๏ธ **Pick a Tool**: Dive into [parser](./parser.md), [linter](./linter.md), [transformer](./transformer.md), or [other tools](./formatter.md) + +We can't wait to see what you'll build with us! ๐Ÿš€ diff --git a/src/docs/contribute/minifier.md b/src/docs/contribute/minifier.md index 6fb3993012..c2a8a6b19b 100644 --- a/src/docs/contribute/minifier.md +++ b/src/docs/contribute/minifier.md @@ -13,12 +13,65 @@ However, existing minifiers typically require a trade-off between compression qu You have to choose between the slowest for the best compression or the fastest for less compression. But what if we could develop a faster minifier without compromising on compression? +## Project Goals + We are actively working on a prototype that aims to achieve this goal, by porting all test cases from well-known minifiers such as [google-closure-compiler], [terser], [esbuild], and [tdewolff-minify]. Preliminary results indicate that we are on track to achieve our objectives. With the Oxc minifier, you can expect faster minification times without sacrificing compression quality. +### Target Performance + +- **Speed**: faster than Terser, competitive with esbuild +- **Compression**: Match or exceed Terser's compression ratio +- **Correctness**: Pass all major minifier test suites + +## Architecture Overview + +### Design Principles + +The Oxc minifier is built around several key principles: + +1. **Semantic-Aware**: Uses semantic analysis to enable safe optimizations +2. **Incremental**: Designed for incremental compilation workflows +3. **Configurable**: Supports various optimization levels and targets +4. **Correct**: Prioritizes correctness over aggressive optimization + +## Current Status + +### Implemented Features + +- โœ… **Dead Code Elimination**: Remove unreachable code +- โœ… **Constant Folding**: Evaluate constant expressions +- โœ… **Tree Shaking**: Remove unused exports (basic) +- โœ… **Variable Merging**: Merge variable declarations +- โœ… **Statement Merging**: Combine compatible statements +- โœ… **Name Mangling**: Shorten variable and function names +- โœ… **Control Flow Optimization**: Simplify control structures +- โœ… **Function Inlining**: Inline small functions +- โœ… **Advanced Tree Shaking**: Cross-module optimization + +### Performance Optimization + +Key strategies for maintaining performance: + +1. **Minimal AST Traversals**: Combine multiple optimizations in single passes +2. **Efficient Data Structures**: Use arena allocation and compact representations +3. **Early Termination**: Skip optimizations when no benefit is possible + +## Resources + +### Documentation + +- [Minifier API Documentation](https://docs.rs/oxc_minifier) + +### External References + +- [Google Closure Compiler Optimizations](https://github.com/google/closure-compiler/wiki/JS-Modules) +- [Terser Options](https://github.com/terser/terser#minify-options) +- [esbuild Minification](https://esbuild.github.io/api/#minification) + [google-closure-compiler]: https://github.com/google/closure-compiler [terser]: https://github.com/terser/terser [esbuild]: https://github.com/evanw/esbuild diff --git a/src/docs/contribute/parser.md b/src/docs/contribute/parser.md index 1b6fe38831..b39d1d5161 100644 --- a/src/docs/contribute/parser.md +++ b/src/docs/contribute/parser.md @@ -5,27 +5,103 @@ outline: deep # Parser -We aim to be the fastest Rust-based ready-for-production parser. +The Oxc parser is designed to be the fastest and most conformant JavaScript and TypeScript parser available. Contributing to the parser requires understanding both the implementation details and the extensive test infrastructure. -## Conformance Tests +## Architecture Overview + +The parser follows a traditional compiler frontend architecture: + +``` +Source Text โ†’ Lexer โ†’ Tokens โ†’ Parser โ†’ AST +``` + +### Key Components + +- **Lexer**: Tokenizes source text into structured tokens +- **Parser**: Recursive descent parser that builds the AST +- **AST**: Memory-efficient abstract syntax tree +- **Error Recovery**: Advanced error handling and recovery +- **Semantic Analysis**: Symbol resolution and scope management + +### Design Goals + +We aim to be the fastest Rust-based ready-for-production parser with: + +- **Speed**: 3x faster than SWC, 5x faster than Biome +- **Conformance**: 100% Test262 compliance, 99%+ Babel/TypeScript compatibility +- **Memory Efficiency**: Arena-based allocation, minimal heap usage +- **Error Quality**: Helpful error messages with recovery + +## Development Workflow + +### Setting Up + +```bash +# Run parser tests +cargo test -p oxc_parser + +# Run conformance tests +just c # or `just coverage` +``` + +### Project Structure + +``` +crates/oxc_parser/ +โ”œโ”€โ”€ src/ +โ”‚ โ”œโ”€โ”€ lib.rs # Public API +โ”‚ โ”œโ”€โ”€ lexer/ # Tokenization +โ”‚ โ”œโ”€โ”€ parser/ # Parsing logic +โ”‚ โ”œโ”€โ”€ cursor.rs # Token stream management +โ”‚ โ””โ”€โ”€ diagnostics.rs # Error handling +โ”œโ”€โ”€ tests/ # Unit tests +โ””โ”€โ”€ examples/ # Usage examples +``` + +### Core Parser Files + +- **`parser/mod.rs`**: Main parser entry point +- **`parser/statement.rs`**: Statement parsing +- **`parser/expression.rs`**: Expression parsing +- **`parser/typescript.rs`**: TypeScript-specific parsing +- **`parser/jsx.rs`**: JSX parsing logic + +## Conformance Testing + +### Running Conformance Tests ```bash just c ``` -Aliased to `just coverage`, runs the following conformance test suites by using the conformance runner found in [tasks/coverage](https://github.com/oxc-project/oxc/tree/main/tasks/coverage). +This runs conformance test suites using the runner in [tasks/coverage](https://github.com/oxc-project/oxc/tree/main/tasks/coverage): -### Test262 +### Test262 - ECMAScript Conformance JavaScript has the [ECMAScript Test Suite](https://github.com/tc39/test262) called Test262. The goal of Test262 is to provide test material that covers every observable behavior specified in the specification. + Parser conformance uses the [parse phase tests](https://github.com/tc39/test262/blob/main/INTERPRETING.md#negative). -### Babel +**Current Status**: `43765/43765 (100.00%)` + +### Babel Parser Tests -When new language features are added to JavaScript, it is required to have them implemented by Babel, -this means Babel has another set of [parser tests](https://github.com/babel/babel/tree/main/packages/babel-parser/test). +When new language features are added to JavaScript, Babel implements them first. +Babel has comprehensive [parser tests](https://github.com/babel/babel/tree/main/packages/babel-parser/test) for cutting-edge features. -### TypeScript +**Current Status**: `2093/2101 (99.62%)` + +### TypeScript Conformance The TypeScript conformance tests can be found [here](https://github.com/microsoft/TypeScript/tree/main/tests/cases/conformance). + +**Current Status**: `6470/6479 (99.86%)` + +### Viewing Results + +Test results are stored in snapshot files for tracking changes: + +- [`parser_test262.snap`](https://github.com/oxc-project/oxc/blob/main/tasks/coverage/parser_test262.snap) +- [`parser_babel.snap`](https://github.com/oxc-project/oxc/blob/main/tasks/coverage/parser_babel.snap) +- [`parser_typescript.snap`](https://github.com/oxc-project/oxc/blob/main/tasks/coverage/parser_typescript.snap) diff --git a/src/docs/contribute/parser/ast.md b/src/docs/contribute/parser/ast.md index 944d6ab9b1..8b4453bcba 100644 --- a/src/docs/contribute/parser/ast.md +++ b/src/docs/contribute/parser/ast.md @@ -3,12 +3,301 @@ title: AST outline: deep --- -# AST +# Abstract Syntax Tree (AST) -## Generate AST related code +The Oxc AST is the foundation of all Oxc tools. Understanding its structure and how to work with it is essential for contributing to parser, linter, transformer, and other components. -Run `just ast` to generate all AST related boilerplate code when AST is changed. +## AST Architecture -## Compare with other AST formats +### Design Principles -- Use [ast-explorer.dev](https://ast-explorer.dev), which has better UI and update to date versions compared to [astexplorer.net](https://ast-explorer.dev). +The Oxc AST is designed with the following principles: + +1. **Performance First**: Optimized for speed and memory efficiency +2. **Type Safety**: Leverages Rust's type system to prevent common errors +3. **Spec Compliance**: Closely follows ECMAScript specification +4. **Clear Semantics**: Removes ambiguity present in other AST formats + +## Working with the AST + +### Generate AST Related Code + +When you modify AST definitions, run the code generation tool: + +```bash +just ast +``` + +This generates: + +- **Visitor patterns**: For traversing the AST +- **Builder methods**: For constructing AST nodes +- **Trait implementations**: For common operations +- **TypeScript types**: For Node.js bindings + +### AST Node Structure + +Every AST node follows a consistent pattern: + +```rust +#[ast(visit)] +pub struct FunctionDeclaration<'a> { + pub span: Span, + pub id: Option>, + pub generator: bool, + pub r#async: bool, + pub params: FormalParameters<'a>, + pub body: Option>, + pub type_parameters: Option>, + pub return_type: Option>, +} +``` + +Key components: + +- **`span`**: Source location information +- **`#[ast(visit)]`**: Generates visitor methods +- **Lifetime `'a`**: References to arena-allocated memory + +### Memory Management + +The AST uses a memory arena for efficient allocation: + +```rust +use oxc_allocator::Allocator; + +let allocator = Allocator::default(); +let ast = parser.parse(&allocator, source_text, source_type)?; +``` + +Benefits: + +- **Fast allocation**: No individual malloc calls +- **Fast deallocation**: Drop entire arena at once +- **Cache friendly**: Linear memory layout +- **No reference counting**: Simple lifetime management + +## AST Traversal + +### Visitor Pattern + +Use the generated visitor for AST traversal: + +```rust +use oxc_ast::visit::{Visit, walk_mut}; + +struct MyVisitor; + +impl<'a> Visit<'a> for MyVisitor { + fn visit_function_declaration(&mut self, func: &FunctionDeclaration<'a>) { + println!("Found function: {:?}", func.id); + walk_mut::walk_function_declaration(self, func); + } +} + +// Usage +let mut visitor = MyVisitor; +visitor.visit_program(&program); +``` + +### Mutable Visitor + +For transformations, use the mutable visitor: + +```rust +use oxc_ast::visit::{VisitMut, walk_mut}; + +struct MyTransformer; + +impl<'a> VisitMut<'a> for MyTransformer { + fn visit_binary_expression(&mut self, expr: &mut BinaryExpression<'a>) { + // Transform the expression + if expr.operator == BinaryOperator::Addition { + // Modify the AST node + } + walk_mut::walk_binary_expression_mut(self, expr); + } +} +``` + +## AST Construction + +### Builder Pattern + +Use the AST builder for creating nodes: + +```rust +use oxc_ast::AstBuilder; + +let ast = AstBuilder::new(&allocator); + +// Create a binary expression: a + b +let left = ast.expression_identifier_reference(SPAN, "a"); +let right = ast.expression_identifier_reference(SPAN, "b"); +let expr = ast.expression_binary_expression( + SPAN, + left, + BinaryOperator::Addition, + right, +); +``` + +### Helper Functions + +Common patterns are provided as helpers: + +```rust +impl<'a> AstBuilder<'a> { + pub fn expression_number_literal(&self, span: Span, value: f64) -> Expression<'a> { + self.alloc(Expression::NumericLiteral( + self.alloc(NumericLiteral { span, value, raw: None }) + )) + } +} +``` + +## Development Workflow + +### Adding New AST Nodes + +1. **Define the struct**: + ```rust + #[ast(visit)] + pub struct MyNewNode<'a> { + pub span: Span, + pub name: Atom<'a>, + pub value: Expression<'a>, + } + ``` + +2. **Add to enum**: + ```rust + pub enum Statement<'a> { + // ... existing variants + MyNewStatement(Box<'a, MyNewNode<'a>>), + } + ``` + +3. **Run code generation**: + ```bash + just ast + ``` + +4. **Implement parsing logic**: + ```rust + impl<'a> Parser<'a> { + fn parse_my_new_node(&mut self) -> Result> { + // Parsing implementation + } + } + ``` + +## Comparing AST Formats + +### Use AST Explorer + +For comparing with other parsers, use [ast-explorer.dev](https://ast-explorer.dev): + +1. **Better UI**: Modern interface with syntax highlighting +2. **Up-to-date**: Latest parser versions +3. **Multiple parsers**: Compare Oxc, Babel, TypeScript, etc. +4. **Export formats**: JSON, code generation + +## Performance Considerations + +### Memory Layout + +The AST is designed for cache efficiency: + +```rust +// Good: Compact representation +struct CompactNode<'a> { + span: Span, // 8 bytes + flags: u8, // 1 byte + name: Atom<'a>, // 8 bytes +} + +// Avoid: Large enums without boxing +enum LargeEnum { + Small, + Large { /* 200 bytes of data */ }, +} +``` + +### Arena Allocation + +All AST nodes are allocated in the arena: + +```rust +// Automatically handled by #[ast] macro +let node = self.ast.alloc(MyNode { + span: SPAN, + value: 42, +}); +``` + +### Enum Size Testing + +We enforce small enum sizes: + +```rust +#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))] +#[test] +fn no_bloat_enum_sizes() { + use std::mem::size_of; + assert_eq!(size_of::(), 16); + assert_eq!(size_of::(), 16); + assert_eq!(size_of::(), 16); +} +``` + +## Advanced Topics + +### Custom AST Attributes + +Add custom attributes for specific tools: + +```rust +#[ast(visit)] +#[cfg_attr(feature = "serialize", derive(Serialize))] +pub struct MyNode<'a> { + #[cfg_attr(feature = "serialize", serde(skip))] + pub internal_data: u32, + pub public_field: Atom<'a>, +} +``` + +### Integration with Semantic Analysis + +Link AST nodes with semantic information: + +```rust +#[ast(visit)] +pub struct IdentifierReference<'a> { + pub span: Span, + pub name: Atom<'a>, + #[ast(ignore)] + pub reference_id: Cell>, +} +``` + +This allows tools to access binding information, scope context, and type information during AST traversal. + +## Debugging Tips + +### Pretty Printing + +Use the debug formatter to inspect AST: + +```rust +println!("{:#?}", ast_node); +``` + +### Span Information + +Track source locations for error reporting: + +```rust +let span = node.span(); +println!("Error at {}:{}", span.start, span.end); +``` diff --git a/src/docs/contribute/resolver.md b/src/docs/contribute/resolver.md index abcdee4142..bd8a74fbb0 100644 --- a/src/docs/contribute/resolver.md +++ b/src/docs/contribute/resolver.md @@ -5,6 +5,13 @@ outline: deep # Resolver -The resolver is in [its own GitHub repository](https://github.com/oxc-project/oxc_resolver). +The Oxc resolver is a high-performance Node.js module resolution implementation that's compatible with webpack's enhanced-resolve. It's maintained in [its own GitHub repository](https://github.com/oxc-project/oxc_resolver). -The APIs are a direct port of [enhanced-resolve](https://github.com/webpack/enhanced-resolve). +## Architecture + +The resolver is designed as a direct port of [enhanced-resolve](https://github.com/webpack/enhanced-resolve) with significant performance improvements: + +- **28x faster** than enhanced-resolve +- **Zero-copy string operations** where possible +- **Optimized path traversal** algorithms +- **Efficient caching** strategies diff --git a/src/docs/contribute/transformer.md b/src/docs/contribute/transformer.md index 6db1bbaa4d..e6f0952f46 100644 --- a/src/docs/contribute/transformer.md +++ b/src/docs/contribute/transformer.md @@ -5,8 +5,21 @@ outline: deep # Transformer -A transformer is responsible for turning higher versions of ECMAScript to a lower version that can be used in older browsers. +The Oxc transformer is responsible for converting higher versions of ECMAScript and TypeScript to lower versions that can run in older browsers and environments. -We are currently focusing on an esnext to es2015 transpiler. +### Repository Structure -See the [umbrella issue](https://github.com/oxc-project/oxc/issues/974) for details. +``` +crates/oxc_transformer/ +โ”œโ”€โ”€ src/ +โ”‚ โ”œโ”€โ”€ lib.rs # Main transformer interface +โ”‚ โ”œโ”€โ”€ transformer.rs # Core transformation logic +โ”‚ โ”œโ”€โ”€ typescript/ # TypeScript transformations +โ”‚ โ”œโ”€โ”€ jsx/ # JSX transformations +โ”‚ โ”œโ”€โ”€ es2015/ # ES2015+ transformations +โ”‚ โ”œโ”€โ”€ isolated_declarations/ # .d.ts generation +โ”‚ โ””โ”€โ”€ helpers/ # Utility functions +โ”œโ”€โ”€ tests/ # Integration tests +โ”œโ”€โ”€ examples/ # Usage examples +โ””โ”€โ”€ benches/ # Performance benchmarks +``` diff --git a/src/docs/guide/usage/linter/rules/nested-config.md b/src/docs/guide/usage/linter/rules/nested-config.md deleted file mode 100644 index e69de29bb2..0000000000 diff --git a/src/docs/learn/architecture/parser.md b/src/docs/learn/architecture/parser.md index bf51cd0ba2..72ce63bb37 100644 --- a/src/docs/learn/architecture/parser.md +++ b/src/docs/learn/architecture/parser.md @@ -3,7 +3,7 @@ title: Parser outline: deep --- -# Parser +# Parser Architecture Oxc maintains its own AST and parser, which is by far the fastest and most conformant JavaScript and TypeScript (including JSX and TSX) parser written in Rust. @@ -11,19 +11,166 @@ As the parser often represents a key performance bottleneck in JavaScript toolin any minor improvements can have a cascading effect on our downstream tools. By developing our parser, we have the opportunity to explore and implement well-researched performance techniques. +## AST Design Philosophy + While many existing JavaScript tools rely on [estree] as their AST specification, a notable drawback is its abundance of ambiguous nodes. This ambiguity often leads to confusion during development with [estree]. -The Oxc AST differs slightly from the [estree] AST by removing ambiguous nodes and introducing distinct types. +The Oxc AST differs from the [estree] AST by removing ambiguous nodes and introducing distinct types. For example, instead of using a generic [estree] `Identifier`, the Oxc AST provides specific types such as `BindingIdentifier`, `IdentifierReference`, and `IdentifierName`. This clear distinction greatly enhances the development experience by aligning more closely with the ECMAScript specification. -## How is it so fast +### AST Node Types + +```rust +// Instead of generic Identifier +pub struct BindingIdentifier<'a> { + pub span: Span, + pub name: Atom<'a>, +} + +pub struct IdentifierReference<'a> { + pub span: Span, + pub name: Atom<'a>, + pub reference_id: Cell>, +} + +pub struct IdentifierName<'a> { + pub span: Span, + pub name: Atom<'a>, +} +``` + +### Semantic Clarity + +This approach provides semantic clarity: + +- **`BindingIdentifier`**: Variable declarations (`let x = 1`) +- **`IdentifierReference`**: Variable usage (`console.log(x)`) +- **`IdentifierName`**: Property names (`obj.property`) + +## Performance Architecture + +### How is it so fast + +- **Memory Arena**: AST is allocated in a [memory arena](https://crates.io/crates/bumpalo) for fast allocation and deallocation +- **String Optimization**: Short strings are inlined by [CompactString](https://crates.io/crates/compact_str) +- **Minimal Heap Usage**: No other heap allocations are done except the above two +- **Separation of Concerns**: Scope binding, symbol resolution and some syntax errors are delegated to the semantic analyzer + +### Memory Management Details + +#### Arena Allocation + +```rust +use oxc_allocator::Allocator; + +// All AST nodes are allocated in this arena +let allocator = Allocator::default(); +let ast_node = allocator.alloc(Expression::NumericLiteral( + allocator.alloc(NumericLiteral { value: 42.0, span: SPAN }) +)); +``` + +Benefits: + +- **O(1) allocation**: Simple pointer bump +- **O(1) deallocation**: Drop entire arena at once +- **Cache-friendly**: Linear memory layout +- **No fragmentation**: Continuous memory usage + +#### String Interning with CompactString + +```rust +// Strings โ‰ค 24 bytes are stored inline (no heap allocation) +let short_name = CompactString::from("variableName"); // Stack allocated +let long_name = CompactString::from("a_very_long_variable_name_that_exceeds_limit"); // Heap allocated +``` + +This reduces memory allocations for the majority of JavaScript identifiers and string literals. + +## Parser Architecture + +### Two-Phase Design + +The Oxc parser follows a two-phase approach: + +1. **Parsing Phase**: Build AST structure with minimal semantic analysis +2. **Semantic Phase**: Perform scope analysis, symbol resolution, and advanced error checking + +```rust +// Phase 1: Parse to AST +let parser_result = Parser::new(&allocator, source_text, source_type).parse(); + +// Phase 2: Semantic analysis +let semantic_result = SemanticBuilder::new(source_text, source_type) + .with_trivias(parser_result.trivias) + .build(&parser_result.program); +``` + +### Parser Components + +#### Lexer + +- **Token generation**: Converts source text to structured tokens +- **SIMD optimization**: Uses SIMD instructions for whitespace skipping +- **Context-aware**: Handles regex vs division operator disambiguation + +#### Recursive Descent Parser + +- **Hand-written**: Custom implementation for maximum performance +- **Error recovery**: Advanced error handling with meaningful messages +- **Grammar compliance**: Follows ECMAScript specification precisely + +#### AST Builder + +- **Type safety**: Leverages Rust's type system for correctness +- **Memory efficiency**: Direct arena allocation +- **Builder pattern**: Convenient node construction methods + +## Conformance Strategy + +### Test Suite Coverage + +- **Test262**: 100% pass rate on ECMAScript conformance tests +- **Babel**: 99.62% compatibility with Babel parser tests +- **TypeScript**: 99.86% compatibility with TypeScript compiler tests + +### Error Handling Philosophy + +```rust +// Meaningful error messages with source location +pub struct OxcDiagnostic { + pub message: String, + pub span: Span, + pub severity: Severity, + pub help: Option, +} +``` + +The parser provides: + +- **Precise error locations**: Exact source positions +- **Recovery strategies**: Continue parsing after errors +- **Helpful suggestions**: Actionable error messages + +## Advanced Features + +### TypeScript Support + +- **Type stripping**: Removes TypeScript-specific syntax +- **Decorator parsing**: Handles experimental decorators +- **Namespace support**: Full module and namespace parsing +- **JSX integration**: TypeScript + JSX (TSX) support + +### Research Areas + +- **SIMD text processing**: Vectorized string operations +- **Cache optimization**: Minimize memory access patterns +- **Branch prediction**: Optimize hot parsing paths +- **Zero-copy parsing**: Eliminate unnecessary string copies -- AST is allocated in a [memory arena](https://crates.io/crates/bumpalo) for fast AST memory allocation and deallocation -- Short strings are inlined by [CompactString](https://crates.io/crates/compact_str) -- No other heap allocations are done except the above two -- Scope binding, symbol resolution and some syntax errors are not done in the parser, they are delegated to the semantic analyzer +[estree]: https://github.com/estree/estree