Skip to content

test: add test profiling script#7086

Open
Bertie690 wants to merge 15 commits intopagefaultgames:betafrom
Bertie690:flamegraph
Open

test: add test profiling script#7086
Bertie690 wants to merge 15 commits intopagefaultgames:betafrom
Bertie690:flamegraph

Conversation

@Bertie690
Copy link
Contributor

@Bertie690 Bertie690 commented Feb 19, 2026

What are the changes the user will see?

N/A

Why am I making these changes?

Makes it much easier to debug memory and performance bottlenecks in code.

What are the changes from a developer perspective?

Added a new script, pnpm test:profile, that instruments a Vitest test run, processes the generated profiler logs, and writes the results to disk to be viewed in a flamegraph tool.
I took the liberty of adding commander.js and its type-checked cousin as a dev dependency (which we really should be using, given how much it streamlines the whole process.argv pipeline)

Note

Given VS Code natively supports opening .cpuprofile and .memprofile files as a flamegraph, I saw it unnecessary to use an external tool to visualize them. (Also because I couldn't get it to work correctly.)

Screenshots/Videos

image

How to test the changes?

Run pnpm test:profile, then open the results in your flamegraph tool of choice. (VSCode has an extension for this, for what it's worth.)

Checklist

  • The PR content is correctly formatted:
    • I'm using beta as my base branch
    • The current branch is not named beta, main or the name of another long-lived feature branch
    • I have provided a clear explanation of the changes within the PR description
    • The PR title matches the Conventional Commits format (as described in CONTRIBUTING.md)
  • The PR is self-contained and cannot be split into smaller PRs
  • There is no overlap with another open PR
  • The PR has been confirmed to work correctly:
    • I have tested the changes manually
    • The full automated test suite still passes (use pnpm test:silent to test locally)
    • I have created new automated tests (pnpm test:create) or updated existing tests related to the PR's changes if necessary

@Bertie690 Bertie690 requested review from a team as code owners February 19, 2026 05:14
@Bertie690 Bertie690 changed the title test: add test profiling script + commander.js test: add test profiling script Feb 19, 2026
Co-authored-by: NightKev <34855794+DayKev@users.noreply.github.com>
@Bertie690 Bertie690 marked this pull request as draft February 19, 2026 17:29
@Madmadness65 Madmadness65 added the Tests Automated tests related label Feb 21, 2026
@Bertie690 Bertie690 marked this pull request as ready for review February 23, 2026 07:03
Copy link
Member

@SirzBenjie SirzBenjie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth noting that this is of limited usefulness due to our mocks. Most of our memory / performance bottlenecks are going to be due to the canvas and UI processing. That is effectively disabled here. We can optimize the 10%, but the other 90% is probably UI. Regardless, LGTM.

SirzBenjie
SirzBenjie previously approved these changes Mar 1, 2026
@Bertie690
Copy link
Contributor Author

@SirzBenjie I'm more interested in why we got a sudden 50% spike in test duration.
This would help triage any unexpected increases in test time like prior

@Bertie690 Bertie690 requested a review from DayKev March 2, 2026 19:48
@DayKev
Copy link
Collaborator

DayKev commented Mar 4, 2026

So this doesn't allow passing arguments to Vitest, and it doesn't itself pass --silent="passed-only" to Vitest.

Also, I'm getting No CPU or memory profile found! when running it.

@Bertie690
Copy link
Contributor Author

Please elaborate.

You need to make the vitest args the last ones passed to the function.

@DayKev
Copy link
Collaborator

DayKev commented Mar 5, 2026

Please elaborate.

You need to make the vitest args the last ones passed to the function.

PS I:\pokerogue repos\PokeRogue dev> pnpm test:profile tackle --silent

> pokemon-rogue-battle@1.11.13 test:profile I:\pokerogue repos\PokeRogue dev
> node scripts/profiling/profiling.js "tackle" "--silent"

📈 Test Profiler - v1.0.0

error: unknown option '--silent'

Usage: pnpm test:profile [options] <vitest-args...>

Run Vitest with Node's V8 profiler and generate processed profiling output.

Arguments:
  vitest-args          Arguments to pass directly to Vitest.

Options:
  -v, --version        Show the version number.
  -o, --output <path>  Directory to which V8 profiler output will be written. (Default: "./temp/vitest-profile")
  -c, --cpu            Enable CPU profiling. (Default: true)
  --no-cpu             Disable CPU profiling.
  -m, --memory         Enable memory profiling. (Default: true)
  --no-memory          Disable memory profiling.
  --clean              Whether to clean the output directory before writing new profiles.
  -h, --help           Show this help message.
 ELIFECYCLE  Command failed with exit code 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Tests Automated tests related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants