Skip to content

Commit c0aaede

Browse files
authored
Add --warn-error to Rewatch (#7916)
* Add --warn-error to Rewatch * Add Rewatch notes for agents.md * Also pass warnings/errors to parse arguments * Changelog * Don't return warning args for non local deps * fmt * Clean up * extract module_name_package_pairs * Remove additional clone
1 parent 84e45c5 commit c0aaede

File tree

15 files changed

+621
-142
lines changed

15 files changed

+621
-142
lines changed

AGENTS.md

Lines changed: 170 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -254,4 +254,173 @@ The compiler is designed for fast feedback loops and scales to large codebases:
254254
- Remember Lambda IR is the core optimization layer
255255
- All `lam_*.ml` files process this representation
256256
- Use `lam_print.ml` for debugging lambda expressions
257-
- Test both with and without optimization passes
257+
- Test both with and without optimization passes
258+
259+
## Working on the Build System
260+
261+
### Rewatch Architecture
262+
263+
Rewatch is the modern build system written in Rust that replaces the legacy bsb (BuckleScript) build system. It provides faster incremental builds, better error messages, and improved developer experience.
264+
265+
#### Key Components
266+
267+
```
268+
rewatch/src/
269+
├── build/ # Core build system logic
270+
│ ├── build_types.rs # Core data structures (BuildState, Module, etc.)
271+
│ ├── compile.rs # Compilation logic and bsc argument generation
272+
│ ├── parse.rs # AST generation and parser argument handling
273+
│ ├── packages.rs # Package discovery and dependency resolution
274+
│ ├── deps.rs # Dependency analysis and module graph
275+
│ ├── clean.rs # Build artifact cleanup
276+
│ └── logs.rs # Build logging and error reporting
277+
├── cli.rs # Command-line interface definitions
278+
├── config.rs # rescript.json configuration parsing
279+
├── watcher.rs # File watching and incremental builds
280+
└── main.rs # Application entry point
281+
```
282+
283+
#### Build System Flow
284+
285+
1. **Initialization** (`build::initialize_build`)
286+
- Parse `rescript.json` configuration
287+
- Discover packages and dependencies
288+
- Set up compiler information
289+
- Create initial `BuildState`
290+
291+
2. **AST Generation** (`build::parse`)
292+
- Generate AST files using `bsc -bs-ast`
293+
- Handle PPX transformations
294+
- Process JSX
295+
296+
3. **Dependency Analysis** (`build::deps`)
297+
- Analyze module dependencies from AST files
298+
- Build dependency graph
299+
- Detect circular dependencies
300+
301+
4. **Compilation** (`build::compile`)
302+
- Generate `bsc` compiler arguments
303+
- Compile modules in dependency order
304+
- Handle warnings and errors
305+
- Generate JavaScript output
306+
307+
5. **Incremental Updates** (`watcher.rs`)
308+
- Watch for file changes
309+
- Determine dirty modules
310+
- Recompile only affected modules
311+
312+
### Development Guidelines
313+
314+
#### Adding New Features
315+
316+
1. **CLI Arguments**: Add to `cli.rs` in `BuildArgs` and `WatchArgs`
317+
2. **Configuration**: Extend `config.rs` for new `rescript.json` fields
318+
3. **Build Logic**: Modify appropriate `build/*.rs` modules
319+
4. **Thread Parameters**: Pass new parameters through the build system chain
320+
5. **Add Tests**: Include unit tests for new functionality
321+
322+
#### Common Patterns
323+
324+
- **Parameter Threading**: New CLI flags need to be passed through:
325+
- `main.rs``build::build()``initialize_build()``BuildState`
326+
- `main.rs``watcher::start()``async_watch()``initialize_build()`
327+
328+
- **Configuration Precedence**: Command-line flags override `rescript.json` config
329+
- **Error Handling**: Use `anyhow::Result` for error propagation
330+
- **Logging**: Use `log::debug!` for development debugging
331+
332+
#### Testing
333+
334+
```bash
335+
# Run rewatch tests (from project root)
336+
cargo test --manifest-path rewatch/Cargo.toml
337+
338+
# Test specific functionality
339+
cargo test --manifest-path rewatch/Cargo.toml config::tests::test_get_warning_args
340+
341+
# Run clippy for code quality
342+
cargo clippy --manifest-path rewatch/Cargo.toml --all-targets --all-features
343+
344+
# Check formatting
345+
cargo fmt --check --manifest-path rewatch/Cargo.toml
346+
347+
# Build rewatch
348+
cargo build --manifest-path rewatch/Cargo.toml --release
349+
350+
# Or use the Makefile shortcuts
351+
make rewatch # Build rewatch
352+
make test-rewatch # Run integration tests
353+
```
354+
355+
**Note**: The rewatch project is located in the `rewatch/` directory with its own `Cargo.toml` file. All cargo commands should be run from the project root using the `--manifest-path rewatch/Cargo.toml` flag, as shown in the CI workflow.
356+
357+
**Integration Tests**: The `make test-rewatch` command runs bash-based integration tests located in `rewatch/tests/suite-ci.sh`. These tests use the `rewatch/testrepo/` directory as a test workspace with various package configurations to verify rewatch's behavior across different scenarios.
358+
359+
#### Debugging
360+
361+
- **Build State**: Use `log::debug!` to inspect `BuildState` contents
362+
- **Compiler Args**: Check generated `bsc` arguments in `compile.rs`
363+
- **Dependencies**: Inspect module dependency graph in `deps.rs`
364+
- **File Watching**: Monitor file change events in `watcher.rs`
365+
366+
#### Performance Considerations
367+
368+
- **Incremental Builds**: Only recompile dirty modules
369+
- **Parallel Compilation**: Use `rayon` for parallel processing
370+
- **Memory Usage**: Be mindful of `BuildState` size in large projects
371+
- **File I/O**: Minimize file system operations
372+
373+
#### Performance vs Code Quality Trade-offs
374+
375+
When clippy suggests refactoring that could impact performance, consider the trade-offs:
376+
377+
- **Parameter Structs vs Many Arguments**: While clippy prefers parameter structs for functions with many arguments, sometimes the added complexity isn't worth it. Use `#[allow(clippy::too_many_arguments)]` for functions that legitimately need many parameters and where a struct would add unnecessary complexity.
378+
379+
- **Cloning vs Borrowing**: Sometimes cloning is necessary due to Rust's borrow checker rules. If the clone is:
380+
- Small and one-time (e.g., `Vec<String>` with few elements)
381+
- Necessary for correct ownership semantics
382+
- Not in a hot path
383+
384+
Then accept the clone rather than over-engineering the solution.
385+
386+
- **When to Optimize**: Profile before optimizing. Most "performance concerns" in build systems are negligible compared to actual compilation time.
387+
388+
- **Avoid Unnecessary Type Conversions**: When threading parameters through multiple function calls, use consistent types (e.g., `String` throughout) rather than converting between `String` and `&str` at each boundary. This eliminates unnecessary allocations and conversions.
389+
390+
#### Compatibility with Legacy bsb
391+
392+
- **Command-line Flags**: Maintain compatibility with bsb flags where possible
393+
- **Configuration**: Support both old (`bs-*`) and new field names
394+
- **Output Format**: Generate compatible build artifacts
395+
- **Error Messages**: Provide clear migration guidance
396+
397+
### Common Tasks
398+
399+
#### Adding New CLI Flags
400+
401+
1. Add to `BuildArgs` and `WatchArgs` in `cli.rs`
402+
2. Update `From<BuildArgs> for WatchArgs` implementation
403+
3. Pass through `main.rs` to build functions
404+
4. Thread through build system to where it's needed
405+
5. Add unit tests for the new functionality
406+
407+
#### Modifying Compiler Arguments
408+
409+
1. Update `compiler_args()` in `build/compile.rs`
410+
2. Consider both parsing and compilation phases
411+
3. Handle precedence between CLI flags and config
412+
4. Test with various `rescript.json` configurations
413+
414+
#### Working with Dependencies
415+
416+
1. Use `packages.rs` for package discovery
417+
2. Update `deps.rs` for dependency analysis
418+
3. Handle both local and external dependencies
419+
4. Consider dev dependencies vs regular dependencies
420+
421+
#### File Watching
422+
423+
1. Modify `watcher.rs` for file change handling
424+
2. Update `AsyncWatchArgs` for new parameters
425+
3. Handle different file types (`.res`, `.resi`, etc.)
426+
4. Consider performance impact of watching many files

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@
3232
- Option optimization: do not create redundant local vars. https://github.com/rescript-lang/rescript/pull/7915
3333
- Js output: remove superfluous newline after every `if`. https://github.com/rescript-lang/rescript/pull/7920
3434
- Rewatch: Traverse upwards for package resolution in single context projects. https://github.com/rescript-lang/rescript/pull/7896
35+
- Rewatch: Add `--warn-error` flag to `build`. https://github.com/rescript-lang/rescript/pull/7916
3536

3637
#### :house: Internal
3738

rewatch/src/build.rs

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,8 @@ pub fn get_compiler_args(rescript_file_path: &Path) -> Result<String> {
8282
&project_context.current_config,
8383
relative_filename,
8484
&contents,
85+
/* is_local_dep */ true,
86+
/* warn_error_override */ None,
8587
)?;
8688
let is_interface = filename.to_string_lossy().ends_with('i');
8789
let has_interface = if is_interface {
@@ -101,6 +103,7 @@ pub fn get_compiler_args(rescript_file_path: &Path) -> Result<String> {
101103
&None,
102104
is_type_dev,
103105
true,
106+
None, // No warn_error_override for compiler-args command
104107
)?;
105108

106109
let result = serde_json::to_string_pretty(&CompilerArgs {
@@ -132,7 +135,8 @@ pub fn initialize_build(
132135
path: &Path,
133136
build_dev_deps: bool,
134137
snapshot_output: bool,
135-
) -> Result<BuildState> {
138+
warn_error: Option<String>,
139+
) -> Result<BuildCommandState> {
136140
let project_context = ProjectContext::new(path)?;
137141
let compiler = get_compiler_info(&project_context)?;
138142

@@ -189,7 +193,7 @@ pub fn initialize_build(
189193
let _ = stdout().flush();
190194
}
191195

192-
let mut build_state = BuildState::new(project_context, packages, compiler);
196+
let mut build_state = BuildCommandState::new(project_context, packages, compiler, warn_error);
193197
packages::parse_packages(&mut build_state);
194198
let timing_source_files_elapsed = timing_source_files.elapsed();
195199

@@ -300,7 +304,7 @@ impl fmt::Display for IncrementalBuildError {
300304
}
301305

302306
pub fn incremental_build(
303-
build_state: &mut BuildState,
307+
build_state: &mut BuildCommandState,
304308
default_timing: Option<Duration>,
305309
initial_build: bool,
306310
show_progress: bool,
@@ -370,7 +374,8 @@ pub fn incremental_build(
370374
}
371375
}
372376
let timing_deps = Instant::now();
373-
deps::get_deps(build_state, &build_state.deleted_modules.to_owned());
377+
let deleted_modules = build_state.deleted_modules.clone();
378+
deps::get_deps(build_state, &deleted_modules);
374379
let timing_deps_elapsed = timing_deps.elapsed();
375380
current_step += 1;
376381

@@ -385,7 +390,7 @@ pub fn incremental_build(
385390
}
386391

387392
mark_modules_with_expired_deps_dirty(build_state);
388-
mark_modules_with_deleted_deps_dirty(build_state);
393+
mark_modules_with_deleted_deps_dirty(&mut build_state.build_state);
389394
current_step += 1;
390395

391396
//print all the compile_dirty modules
@@ -488,7 +493,7 @@ pub fn incremental_build(
488493
}
489494
}
490495

491-
fn log_deprecations(build_state: &BuildState) {
496+
fn log_deprecations(build_state: &BuildCommandState) {
492497
build_state.packages.iter().for_each(|(_, package)| {
493498
// Only warn for local dependencies, not external packages
494499
if package.is_local_dep {
@@ -522,14 +527,15 @@ fn log_deprecated_config_field(package_name: &str, field_name: &str, new_field_n
522527
// is watching this file.
523528
// we don't need to do this in an incremental build because there are no file
524529
// changes (deletes / additions)
525-
pub fn write_build_ninja(build_state: &BuildState) {
530+
pub fn write_build_ninja(build_state: &BuildCommandState) {
526531
for package in build_state.packages.values() {
527532
// write empty file:
528533
let mut f = File::create(package.get_build_path().join("build.ninja")).expect("Unable to write file");
529534
f.write_all(b"").expect("unable to write to ninja file");
530535
}
531536
}
532537

538+
#[allow(clippy::too_many_arguments)]
533539
pub fn build(
534540
filter: &Option<regex::Regex>,
535541
path: &Path,
@@ -538,7 +544,8 @@ pub fn build(
538544
create_sourcedirs: bool,
539545
build_dev_deps: bool,
540546
snapshot_output: bool,
541-
) -> Result<BuildState> {
547+
warn_error: Option<String>,
548+
) -> Result<BuildCommandState> {
542549
let default_timing: Option<std::time::Duration> = if no_timing {
543550
Some(std::time::Duration::new(0.0 as u64, 0.0 as u32))
544551
} else {
@@ -552,6 +559,7 @@ pub fn build(
552559
path,
553560
build_dev_deps,
554561
snapshot_output,
562+
warn_error,
555563
)
556564
.map_err(|e| anyhow!("Could not initialize build. Error: {e}"))?;
557565

rewatch/src/build/build_types.rs

Lines changed: 62 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ use crate::config::Config;
33
use crate::project_context::ProjectContext;
44
use ahash::{AHashMap, AHashSet};
55
use blake3::Hash;
6-
use std::{fmt::Display, path::PathBuf, time::SystemTime};
6+
use std::{fmt::Display, ops::Deref, path::PathBuf, time::SystemTime};
77

88
#[derive(Debug, Clone, PartialEq)]
99
pub enum ParseState {
@@ -90,6 +90,9 @@ impl Module {
9090
}
9191
}
9292

93+
/// Core build state containing all the essential data needed for compilation.
94+
/// This is the minimal state required for basic build operations like cleaning.
95+
/// Used by commands that don't need command-line specific overrides (e.g., `clean`).
9396
#[derive(Debug)]
9497
pub struct BuildState {
9598
pub project_context: ProjectContext,
@@ -101,6 +104,21 @@ pub struct BuildState {
101104
pub deps_initialized: bool,
102105
}
103106

107+
/// Extended build state that includes command-line specific overrides.
108+
/// Wraps `BuildState` and adds command-specific data like warning overrides.
109+
/// Used by commands that need to respect CLI flags (e.g., `build`, `watch`).
110+
///
111+
/// The separation exists because:
112+
/// - `clean` command only needs core build data, no CLI overrides
113+
/// - `build`/`watch` commands need both core data AND CLI overrides
114+
/// - This prevents the "code smell" of optional fields that are None for some commands
115+
#[derive(Debug)]
116+
pub struct BuildCommandState {
117+
pub build_state: BuildState,
118+
// Command-line --warn-error flag override (takes precedence over rescript.json config)
119+
pub warn_error_override: Option<String>,
120+
}
121+
104122
#[derive(Debug, Clone)]
105123
pub struct CompilerInfo {
106124
pub bsc_path: PathBuf,
@@ -116,6 +134,7 @@ impl BuildState {
116134
pub fn get_module(&self, module_name: &str) -> Option<&Module> {
117135
self.modules.get(module_name)
118136
}
137+
119138
pub fn new(
120139
project_context: ProjectContext,
121140
packages: AHashMap<String, Package>,
@@ -142,6 +161,48 @@ impl BuildState {
142161
}
143162
}
144163

164+
impl BuildCommandState {
165+
pub fn new(
166+
project_context: ProjectContext,
167+
packages: AHashMap<String, Package>,
168+
compiler: CompilerInfo,
169+
warn_error_override: Option<String>,
170+
) -> Self {
171+
Self {
172+
build_state: BuildState::new(project_context, packages, compiler),
173+
warn_error_override,
174+
}
175+
}
176+
177+
pub fn get_warn_error_override(&self) -> Option<String> {
178+
self.warn_error_override.clone()
179+
}
180+
181+
pub fn module_name_package_pairs(&self) -> Vec<(String, String)> {
182+
self.build_state
183+
.modules
184+
.iter()
185+
.map(|(name, module)| (name.clone(), module.package_name.clone()))
186+
.collect()
187+
}
188+
}
189+
190+
// Implement Deref to automatically delegate method calls to the inner BuildState
191+
impl Deref for BuildCommandState {
192+
type Target = BuildState;
193+
194+
fn deref(&self) -> &Self::Target {
195+
&self.build_state
196+
}
197+
}
198+
199+
// Implement DerefMut to allow mutable access to the inner BuildState
200+
impl std::ops::DerefMut for BuildCommandState {
201+
fn deref_mut(&mut self) -> &mut Self::Target {
202+
&mut self.build_state
203+
}
204+
}
205+
145206
#[derive(Debug)]
146207
pub struct AstModule {
147208
pub module_name: String,

0 commit comments

Comments
 (0)