Skip to content

tools: port dust a source module minimizer#979

Merged
chore-runner[bot] merged 58 commits intonim-works:develfrom
saem:saem-tool-dust-port
Feb 16, 2026
Merged

tools: port dust a source module minimizer#979
chore-runner[bot] merged 58 commits intonim-works:develfrom
saem:saem-tool-dust-port

Conversation

@saem
Copy link
Collaborator

@saem saem commented Oct 19, 2023

Summary

Add a compiler error message minimizer executable to the compiler
distribution, called dust . Given a compiler error message,
dust $compiler_options $module_to_reduce will reduce said module's
code in place until a minimal module that generates the same error is
produced.

Details

dust is a new executable that ships with NimSkull, that minimizes a
module down to the smallest failing module. This is useful for those
submitting bugs, allowing for easy reduction in test cases size, or
those looking to reduce failing code to better understand it.

It can be run by calling dust <file_to_minimize.nim> , it will then
reduce the file to the minimal failing case. This is an inplace edit --
so be sure to save a copy if you want to keep the original.

In addition to the introduction of dust , a dedicated tester also
exists meant to facilitate testing of the tool, it's a simple driver for
the dust executable.

Porting Details

dust was originally written by @disruptek, this ported version has a
few key differences:

  1. It works on NimSkull and ships with the compiler, ensuring we're
    always reducing with a like for like compiler
  2. The search algorithm and AST mutators are much faster (greater than
    70x)
  3. Includes a tester that can be run via CI

Copy link
Collaborator

@zerbina zerbina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a first pass and left some minor style suggestions.

@saem saem marked this pull request as draft October 20, 2023 19:39
saem and others added 11 commits October 31, 2023 12:22
also there is a bug when it comes to debugging types... sigh
added to to `tools` build in `koch`
Co-authored-by: Clyybber <darkmine956@gmail.com>
Co-authored-by: zerbina <100542850+zerbina@users.noreply.github.com>
Co-authored-by: zerbina <100542850+zerbina@users.noreply.github.com>
Co-authored-by: zerbina <100542850+zerbina@users.noreply.github.com>
Co-authored-by: zerbina <100542850+zerbina@users.noreply.github.com>
@saem saem force-pushed the saem-tool-dust-port branch from 2a5cb71 to 79b0995 Compare October 31, 2023 20:14
saem added 9 commits January 18, 2024 13:12
the problem is that right now we generate more and more errors over
time, so the minimization process diverges, rather than converging.
right now there is a bug in `semcall.semResolvedCall` where a `tyError`
from a faux match doesn't have an error and is being wrapped

I'm guessing this will require removing a bunch of unwrapping logic to
fix, but not quite sure.
Wrapping nodes is no longer necesssary as they're reported immediately.
This was resulting in attempting to wrap nodes that no longer contain
errors but resulted in them.
@saem
Copy link
Collaborator Author

saem commented Oct 13, 2025

Dust now works and successfully reduces the broken code in mminimizeme.nim, it's a bit slow, but that's probably fixable as sem gets faster (removal of unnecessary wrapping etc).

Another thing that will likely help a fair bit is a breadth first search and early elimination of the search space, but I'm not sure how to implement that (besides the breadth first) just yet.

saem added 2 commits December 23, 2025 10:36
This was a misunderstanding of the issue and the fix isn't required (it'll actually likely cause issues).
@saem saem changed the title tools: added a port of dust tools: port dust a test case minimizer Feb 7, 2026
@github-actions
Copy link

Merge requested by: @saem

Contents after the first section break of the PR description has been removed and preserved below:


Notes for Reviewers

  • create a proper PR message
  • figure out some minor testing, at least try it
  • audit dust code to use the latest in nimskull compiler
  • remove warnings and hints
  • undo the error wrap change as it leads to issues (e.g. unlimited template expansion)
  • add dust as a release artifact
  • merge all imports into a single section at the top of the file
  • remove the spaces after and before / in import paths
  • add a small doc comment at the top of each new module, detailing their purpose
  • change the procedure-level doc comments to use third-person singular and to start with a capital letter and end with a period
  • add relevant docs? (suggestions from others welcome)

@chore-runner chore-runner bot added this pull request to the merge queue Feb 14, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 14, 2026
@saem
Copy link
Collaborator Author

saem commented Feb 14, 2026

/merge

@chore-runner chore-runner bot added this pull request to the merge queue Feb 14, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 14, 2026
@saem
Copy link
Collaborator Author

saem commented Feb 15, 2026

/merge

@chore-runner chore-runner bot added this pull request to the merge queue Feb 15, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 15, 2026
@saem
Copy link
Collaborator Author

saem commented Feb 15, 2026

sigh... I'm guessing it didn't run the leak sanitizer locally when I did ran the tools tests via koch, oh well, I'll have to figure that out and see what to make of the errors.

@saem
Copy link
Collaborator Author

saem commented Feb 15, 2026

I presently can't get a local reproduction, so while I wait to hear back from others about how I might do that I took a stab in the dark and enabled --gc:orc for dust in case that's the issue, on the basis that the vmrunner requires it as well.


Reproduction steps I've tried:

  • install liblsan (previously i got a linker failure until I installed it)
  • compiled dust with all the flags from leak.yml
  • ran koch.py testtools just like leak.yml
  • result: I didn't get any failure spam
  • my os is Fedora 43 workstation, if that helps

@saem
Copy link
Collaborator Author

saem commented Feb 15, 2026

/merge

@chore-runner chore-runner bot enabled auto-merge February 15, 2026 21:14
@chore-runner chore-runner bot added this pull request to the merge queue Feb 15, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 15, 2026
@saem
Copy link
Collaborator Author

saem commented Feb 16, 2026

/merge

@chore-runner chore-runner bot enabled auto-merge February 16, 2026 23:04
@chore-runner chore-runner bot added this pull request to the merge queue Feb 16, 2026
Merged via the queue into nim-works:devel with commit 1d36f68 Feb 16, 2026
77 of 79 checks passed
@saem saem deleted the saem-tool-dust-port branch February 16, 2026 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tool Improvements to non-compiler tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants