Skip to content

Comments

Test alternate storage types for Ruma identifiers#6187

Draft
zecakeh wants to merge 8 commits intomatrix-org:mainfrom
zecakeh:bench-storage
Draft

Test alternate storage types for Ruma identifiers#6187
zecakeh wants to merge 8 commits intomatrix-org:mainfrom
zecakeh:bench-storage

Conversation

@zecakeh
Copy link
Collaborator

@zecakeh zecakeh commented Feb 18, 2026

This is not meant to be merged, I want to see how much changing the inner type of Owned* identifiers influences codspeed benchmarks.

@zecakeh zecakeh changed the title Test Arc storage type Test Arc storage type for Ruma identifiers Feb 18, 2026
@zecakeh zecakeh changed the title Test Arc storage type for Ruma identifiers Test alternate storage type for Ruma identifiers Feb 18, 2026
@codecov
Copy link

codecov bot commented Feb 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.84%. Comparing base (9729e63) to head (154b332).
⚠️ Report is 12 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6187      +/-   ##
==========================================
- Coverage   89.85%   89.84%   -0.01%     
==========================================
  Files         365      365              
  Lines      100845   100845              
  Branches   100845   100845              
==========================================
- Hits        90613    90609       -4     
- Misses       6696     6700       +4     
  Partials     3536     3536              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codspeed-hq
Copy link

codspeed-hq bot commented Feb 18, 2026

Merging this PR will not alter performance

✅ 50 untouched benchmarks
🆕 49 new benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Memory Event cache find_event_relations[memory][10 events, #no filter] N/A 6.1 KB N/A
🆕 Memory Event cache room updates[SQLite][room count: 10] N/A 9.5 MB N/A
🆕 Memory Event cache find_event_relations[memory][1000 events, edits filter] N/A 650.3 KB N/A
🆕 Memory Event cache find_event_relations[memory][100 events, edits filter] N/A 68 KB N/A
🆕 Memory Event cache find_event_relations[memory][1000 events, #no filter] N/A 650.3 KB N/A
🆕 Memory Event cache find_event_relations[SQLite][1000 events, #no filter] N/A 650.3 KB N/A
🆕 Memory Event cache find_event_relations[SQLite][1000 events, edits filter] N/A 650.6 KB N/A
🆕 Memory Event cache find_event_relations[SQLite][100 events, #no filter] N/A 68 KB N/A
🆕 Memory Event cache room updates[memory][room count: 100] N/A 134.4 MB N/A
🆕 Memory Event cache find_event_relations[memory][100 events, #no filter] N/A 68 KB N/A
🆕 Memory Event cache find_event_relations[memory][10 events, edits filter] N/A 6.2 KB N/A
🆕 Memory Event cache room updates[memory][room count: 1] N/A 2.1 MB N/A
🆕 Memory Event cache find_event_relations[SQLite][10 events, edits filter] N/A 12.4 KB N/A
🆕 Memory Event cache find_event_relations[SQLite][10 events, #no filter] N/A 11.8 KB N/A
🆕 Memory Event cache room updates[memory][room count: 10] N/A 14.1 MB N/A
🆕 Memory Event cache room updates[SQLite][room count: 100] N/A 86.3 MB N/A
🆕 Memory Event cache find_event_relations[SQLite][100 events, edits filter] N/A 68 KB N/A
🆕 Memory Event cache room updates[SQLite][room count: 1] N/A 1.9 MB N/A
🆕 Memory Create[1000 rooms × 1000 events] N/A 714.5 KB N/A
🆕 Memory Restore session [SQLite][clear] N/A 65.5 MB N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Comparing zecakeh:bench-storage (154b332) with main (6c18f73)

Open in CodSpeed

@zecakeh zecakeh changed the title Test alternate storage type for Ruma identifiers Test alternate storage types for Ruma identifiers Feb 23, 2026
@zecakeh zecakeh force-pushed the bench-storage branch 6 times, most recently from 0867c87 to b8325bb Compare February 24, 2026 10:31
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Comment on lines +94 to 107
- name: Run the benchmarks in simulation mode
uses: CodSpeedHQ/action@2ac572851726409c88c02a307f1ea2632a9ea59b
with:
run: cargo codspeed run
mode: "instrumentation"
mode: simulation
token: ${{ secrets.CODSPEED_TOKEN }}

- name: Run the benchmarks in memory mode
uses: CodSpeedHQ/action@2ac572851726409c88c02a307f1ea2632a9ea59b
if: ${{ matrix.benchmark != 'timeline' }} # The benchmark fails an assertion, maybe due to it being slower?
with:
run: cargo codspeed run
mode: memory
token: ${{ secrets.CODSPEED_TOKEN }}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, co-founder of @CodSpeedHQ here.

Super glad to have using the memory instrument in addition to simulation.

One thing I recommend though is to use OIDC instead of tokens to authenticate the runs, it will be way faster and more secure 😉

You will need to add the following in your job/workflow:

permissions:
  contents: read # required for actions/checkout
  id-token: write # required for OIDC authentication with CodSpeed

And you will no longer need the token: input of course:

Suggested change
- name: Run the benchmarks in simulation mode
uses: CodSpeedHQ/action@2ac572851726409c88c02a307f1ea2632a9ea59b
with:
run: cargo codspeed run
mode: "instrumentation"
mode: simulation
token: ${{ secrets.CODSPEED_TOKEN }}
- name: Run the benchmarks in memory mode
uses: CodSpeedHQ/action@2ac572851726409c88c02a307f1ea2632a9ea59b
if: ${{ matrix.benchmark != 'timeline' }} # The benchmark fails an assertion, maybe due to it being slower?
with:
run: cargo codspeed run
mode: memory
token: ${{ secrets.CODSPEED_TOKEN }}
- name: Run the benchmarks in simulation mode
uses: CodSpeedHQ/action@2ac572851726409c88c02a307f1ea2632a9ea59b
with:
run: cargo codspeed run
mode: simulation
- name: Run the benchmarks in memory mode
uses: CodSpeedHQ/action@2ac572851726409c88c02a307f1ea2632a9ea59b
if: ${{ matrix.benchmark != 'timeline' }} # The benchmark fails an assertion, maybe due to it being slower?
with:
run: cargo codspeed run
mode: memory

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants