Skip to content

Conversation

seaona
Copy link
Member

@seaona seaona commented Oct 13, 2025

Description

The fill method is flaky as in this spec we can see how the extension id was sent twice, making the spec fail as the dapp fails with Error connecting: TypeError: Error in invocation of runtime.connect(optional string extensionId, optional object connectInfo): Invalid extension id: 'hebhblbkkdabgoldnojllkipeoacjiochebhblbkkdabgoldn

image

Open in GitHub Codespaces

Changelog

CHANGELOG entry:

Related issues

Fixes:

Manual testing steps

  1. Check ci

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Note

Adds a retry/verification option to driver.fill and uses it in the multichain dapp page, plus increases the connect delay to reduce flakiness.

  • E2E Driver (test/e2e/webdriver/driver.js):
    • Enhance fill(rawLocator, input, { retries }) to support optional retries with value verification via getAttribute('value') and waitUntil; throws if not set after attempts. Updates JSDoc accordingly.
  • Multichain Test Dapp Page (test/e2e/page-objects/pages/test-dapp-multichain.ts):
    • Use driver.fill(..., { retries: 3 }) for extensionId input.
    • Increase post-connect wait from largeDelayMs to veryLargeDelayMs.

Written by Cursor Bugbot for commit 1022e62. This will update automatically on new commits. Configure here.

Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@metamaskbot metamaskbot added the team-qa QA team label Oct 13, 2025
@seaona seaona self-assigned this Oct 13, 2025
@seaona seaona added no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed flaky tests and removed size-S labels Oct 13, 2025
@metamaskbot
Copy link
Collaborator

📊 Page Load Benchmark Results

Current Commit: b9ceb76 | Date: 10/13/2025

📄 Localhost MetaMask Test Dapp

Samples: 100

Summary

  • pageLoadTime-> current mean value: 1.07s (±74ms) 🟡 | historical mean value: 1.05s ⬆️ (historical data)
  • domContentLoaded-> current mean value: 752ms (±71ms) 🟢 | historical mean value: 738ms ⬆️ (historical data)
  • firstContentfulPaint-> current mean value: 79ms (±11ms) 🟢 | historical mean value: 77ms ⬆️ (historical data)
📈 Detailed Results
Metric Mean Std Dev Min Max P95 P99
pageLoadTime 1.07s 74ms 1.03s 1.37s 1.32s 1.37s
domContentLoaded 752ms 71ms 713ms 1.03s 981ms 1.03s
firstPaint 79ms 11ms 60ms 168ms 88ms 168ms
firstContentfulPaint 79ms 11ms 60ms 168ms 88ms 168ms
largestContentfulPaint 0ms 0ms 0ms 0ms 0ms 0ms

Results generated automatically by MetaMask CI

@metamaskbot
Copy link
Collaborator

Builds ready [b9ceb76]
UI Startup Metrics (1180 ± 66 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyHomeuiStartup1180103713546612271295
load101389311835910531119
domContentLoaded100788711746010461112
domInteractive18134781744
firstPaint75386115838610391120
backgroundConnect21419424711220237
firstReactRender28185683245
getState1044261122
initialActions50648415
loadScripts79968994758841908
setupStore96253913
WebpackHomeuiStartup821702112171850944
load63658196068643775
domContentLoaded62857595068635758
domInteractive15113971434
firstPaint17053797157186594
backgroundConnect21104162632
firstReactRender2816179183148
getState83203914
initialActions2017336
loadScripts62557393965633744
setupStore84162911
FirefoxBrowserifyHomeuiStartup14471254201114514971693
load12321091158110112951428
domContentLoaded12321090158110112951427
domInteractive1063431259107269
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect3220140173351
firstReactRender3126154133139
getState738110626
initialActions401841928
loadScripts1208106714779812701400
setupStore946510726
WebpackHomeuiStartup15201332190112115861816
load1322119015868713721490
domContentLoaded1322118915858713711490
domInteractive91293474796181
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect3116136153553
firstReactRender362676123674
getState8215418531
initialActions3012813212
loadScripts1297116815578313471456
setupStore9410213733
Bundle size diffs [🚨 Warning! Bundle size has increased!]
  • background: 58 Bytes (0%)
  • ui: 0 Bytes (0%)
  • common: 10 Bytes (0%)

@metamaskbot
Copy link
Collaborator

metamaskbot commented Oct 13, 2025

✨ Files requiring CODEOWNER review ✨

🧪 @MetaMask/qa (1 files, +3 -3)
  • 📁 test/
    • 📁 e2e/
      • 📁 page-objects/
        • 📁 pages/
          • 📄 test-dapp-multichain.ts +3 -3

@metamaskbot
Copy link
Collaborator

📊 Page Load Benchmark Results

Current Commit: aa4d8a0 | Date: 10/13/2025

📄 Localhost MetaMask Test Dapp

Samples: 100

Summary

  • pageLoadTime-> current mean value: 1.04s (±72ms) 🟡 | historical mean value: 1.05s ⬇️ (historical data)
  • domContentLoaded-> current mean value: 730ms (±69ms) 🟢 | historical mean value: 738ms ⬇️ (historical data)
  • firstContentfulPaint-> current mean value: 76ms (±13ms) 🟢 | historical mean value: 77ms ⬇️ (historical data)
📈 Detailed Results
Metric Mean Std Dev Min Max P95 P99
pageLoadTime 1.04s 72ms 1.00s 1.31s 1.29s 1.31s
domContentLoaded 730ms 69ms 694ms 997ms 954ms 997ms
firstPaint 76ms 13ms 56ms 176ms 88ms 176ms
firstContentfulPaint 76ms 13ms 56ms 176ms 88ms 176ms
largestContentfulPaint 0ms 0ms 0ms 0ms 0ms 0ms

Results generated automatically by MetaMask CI

@metamaskbot
Copy link
Collaborator

Builds ready [aa4d8a0]
UI Startup Metrics (1244 ± 63 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyHomeuiStartup1244112615356312771356
load107396013236011111181
domContentLoaded106795513175910991175
domInteractive17144861728
firstPaint62975133543810801149
backgroundConnect2542392878259267
firstReactRender24194742533
getState1156881322
initialActions40203412
loadScripts819717106159855928
setupStore96213916
WebpackHomeuiStartup8297101077748481032
load64358693477640850
domContentLoaded63657992376634840
domInteractive16116791435
firstPaint20657927213180846
backgroundConnect21103972632
firstReactRender26166093144
getState831931013
initialActions2010236
loadScripts63357791274632829
setupStore84162912
FirefoxBrowserifyHomeuiStartup14261224183313314851738
load1217106114338712911357
domContentLoaded1217106114338712901356
domInteractive100342905698240
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect3019106123456
firstReactRender29254533036
getState9323824623
initialActions30193211
loadScripts1194104614038412671332
setupStore11314219758
WebpackHomeuiStartup15401327208412615811789
load1337117516459113751500
domContentLoaded1336117516449113741500
domInteractive912935849101182
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect3517159223589
firstReactRender372678143877
getState8310213636
initialActions3023429
loadScripts1311116015978513441479
setupStore947313737
Bundle size diffs [🚨 Warning! Bundle size has increased!]
  • background: 58 Bytes (0%)
  • ui: 0 Bytes (0%)
  • common: 10 Bytes (0%)

);
await this.clickConnectExternallyConnectableButton();
await this.driver.delay(largeDelayMs);
await this.driver.delay(veryLargeDelayMs);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this intends to mitigate this other error - it seems a large delay wasn't sufficient

image image

@metamaskbot
Copy link
Collaborator

📊 Page Load Benchmark Results

Current Commit: c83017f | Date: 10/13/2025

📄 Localhost MetaMask Test Dapp

Samples: 100

Summary

  • pageLoadTime-> current mean value: 1.08s (±73ms) 🟡 | historical mean value: 1.05s ⬆️ (historical data)
  • domContentLoaded-> current mean value: 755ms (±71ms) 🟢 | historical mean value: 737ms ⬆️ (historical data)
  • firstContentfulPaint-> current mean value: 78ms (±10ms) 🟢 | historical mean value: 77ms ⬆️ (historical data)
📈 Detailed Results
Metric Mean Std Dev Min Max P95 P99
pageLoadTime 1.08s 73ms 1.02s 1.38s 1.28s 1.38s
domContentLoaded 755ms 71ms 708ms 1.06s 960ms 1.06s
firstPaint 78ms 10ms 60ms 164ms 88ms 164ms
firstContentfulPaint 78ms 10ms 60ms 164ms 88ms 164ms
largestContentfulPaint 0ms 0ms 0ms 0ms 0ms 0ms

Results generated automatically by MetaMask CI

@metamaskbot
Copy link
Collaborator

Builds ready [c83017f]
UI Startup Metrics (1250 ± 72 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyHomeuiStartup1250111414407213001383
load107696912396711221207
domContentLoaded106995812336811111202
domInteractive19146791841
firstPaint65672123243310591193
backgroundConnect25323929410257271
firstReactRender27185082844
getState1152751221
initialActions40436513
loadScripts82371699267860961
setupStore1063561025
WebpackHomeuiStartup8146951106708291009
load62857693968637851
domContentLoaded62157093067631842
domInteractive15116481434
firstPaint20158853191192594
backgroundConnect20103952429
firstReactRender26174773134
getState832131012
initialActions207235
loadScripts61856891965629831
setupStore84394911
FirefoxBrowserifyHomeuiStartup15431314214814616021852
load13141127177910613831494
domContentLoaded13131126177710613831493
domInteractive1093636463110279
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect39202362937102
firstReactRender33277063444
getState11319327726
initialActions31163210
loadScripts1285111115629613471469
setupStore947111734
WebpackHomeuiStartup15181355189611115581800
load1323116515717713721478
domContentLoaded1323116515717713721477
domInteractive903024634101160
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect3018127163354
firstReactRender342678113576
getState738211524
initialActions30658217
loadScripts1300115015417313421461
setupStore11413119644
Bundle size diffs [🚀 Bundle size reduced!]
  • background: 48 Bytes (0%)
  • ui: 7 Bytes (0%)
  • common: -27.29 KiB (-0.33%)

@seaona seaona added no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed and removed no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed labels Oct 14, 2025
@metamaskbot
Copy link
Collaborator

📊 Page Load Benchmark Results

Current Commit: 1022e62 | Date: 10/14/2025

📄 Localhost MetaMask Test Dapp

Samples: 100

Summary

  • pageLoadTime-> current mean value: 1.06s (±73ms) 🟡 | historical mean value: 1.05s ⬆️ (historical data)
  • domContentLoaded-> current mean value: 741ms (±70ms) 🟢 | historical mean value: 739ms ⬆️ (historical data)
  • firstContentfulPaint-> current mean value: 77ms (±12ms) 🟢 | historical mean value: 77ms ⬇️ (historical data)
📈 Detailed Results
Metric Mean Std Dev Min Max P95 P99
pageLoadTime 1.06s 73ms 1.02s 1.36s 1.30s 1.36s
domContentLoaded 741ms 70ms 706ms 1.02s 983ms 1.02s
firstPaint 77ms 12ms 60ms 180ms 88ms 180ms
firstContentfulPaint 77ms 12ms 60ms 180ms 88ms 180ms
largestContentfulPaint 0ms 0ms 0ms 0ms 0ms 0ms

Results generated automatically by MetaMask CI

@metamaskbot
Copy link
Collaborator

Builds ready [1022e62]
UI Startup Metrics (1216 ± 64 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyHomeuiStartup1216109113716412611317
load105695111916010951158
domContentLoaded105094611855910901152
domInteractive17134561637
firstPaint721137115941510741122
backgroundConnect2492352777251258
firstReactRender24194352537
getState1053151322
initialActions40738411
loadScripts80771093658847905
setupStore95223918
WebpackHomeuiStartup797682116278812960
load617566100479618818
domContentLoaded61056099878611814
domInteractive14114871333
firstPaint19758937185193578
backgroundConnect19104662332
firstReactRender26166183034
getState73143912
initialActions207236
loadScripts60755898875609803
setupStore84172911
FirefoxBrowserifyHomeuiStartup14261229188812715011630
load1221106514839112981402
domContentLoaded1221106514829112981401
domInteractive1033229952106233
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect3020143153353
firstReactRender29255343140
getState7312012513
initialActions2037429
loadScripts1198104814539012741379
setupStore845810630
WebpackHomeuiStartup15911405224815016571878
load13711206171810414191571
domContentLoaded13711206171710414181571
domInteractive101324787299331
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect3520100183986
firstReactRender392795144176
getState10217024627
initialActions40457224
loadScripts1343119116469513871512
setupStore1149716860
Bundle size diffs [🚀 Bundle size reduced!]
  • background: 48 Bytes (0%)
  • ui: 5.09 KiB (0.08%)
  • common: -27.07 KiB (-0.32%)

@seaona seaona marked this pull request as ready for review October 14, 2025 06:57
@seaona seaona requested a review from a team as a code owner October 14, 2025 06:57
* '0xc427D562164062a23a5cFf596A4a3208e72Acd28');
*/
async fill(rawLocator, input) {
async fill(rawLocator, input, { retries = 0 } = {}) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ I choose to make this as a param with 0 as default, because the verification path can break existing specs (those where the input is then changed after being added (ie send ens domain etc), so it cannot be used everywhere

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

flaky tests no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed size-S team-qa QA team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants