Large increase in random CI failures using Cypress after upgrading to react-router v7

[pjg]

I'm using React Router as a...

library

Reproduction

https://github.com/almilo/react-router-7-ci-bug

There are no reproduction steps. Just an observation. Looking at video of failures, and corresponding code, it looks like a race condition during clicking. The code failing looks similar to this (in various places):

cy.contains('Text').should('exist')
cy.get('.class a').first().click()

The link is on the same page as the text. What happens is that Cypress executes the click, but react-router does not transition to the new path, like the click never happened.

I'm posting this mainly to see if I'm not alone with this.

System Info

react-router v7.1.4
cypress v14.0.1

Used Package Manager

npm

Expected Behavior

Not seeing random test failures.

Actual Behavior

Seeing random test failures.

[timdorr]

Does it transition to the new page if you add some wait time to the test?

We use startTransition between navigations, so this introduces potentially async behavior to rendering. You may either need to wait a tick for the render to finish or have some way to wait for the render to finish before checking the DOM state. That may depend on how you have the test renderer hooked up and what libraries you use to do that.

[pjg]

It's hard to tell as the failures are very random across the whole test suite. I use cypress, and it has some waiting mechanisms built-in, so that there's no need to manually wait. Though they seem to fail occasionally when dealing with react-router v7. I'll keep investigating though.

[dhulme]

I think the issue is that because routing updates are happening in a transition, Cypress or automated testing tools may interact with the UI too quickly before any routing side effects have completed. For example, the below test ensures that the page state and URL stay in the sync. This test now fails unless I ensure that the UI is up to date with the URL before the 'next range' button is clicked.

cy.findByRole('button', { name: /previous range/i }).click();
cy.url().should('include', 'date=2021-04-12');
cy.findByRole('textbox', { name: /date range/i }).should('have.value', '12/04/2021'); // Now fails without this line
cy.findByRole('button', { name: /next range/i }).click();
cy.url().should('include', 'date=2021-04-13');

There are other cases though where the routing state is not visible in the UI, for example a useEffect that updates some state when the page location changes.

Is there a way we can detect if the navigation transition is still in progress, to block any further navigation, or ensure the test waits?

[trainoasis]

Also ran into this. It's related to v7_startTransition part of the upgrade it seems.

PS: managed to get around this by avoiding cy.url() checks at all, but then after actually migrating to v7 a bunch of other issues pop up, like pages not loading in Cypress at all, so most tests fail.

We then realized that the problem was how we build routes: we create them based on some async data dynamically & it seems react-router v7 now doesn't handle this the same as v6 did. To "fix" it we now block rendering until all our routes are stable and then render the Router.

[almilo]

I am working on a project where we use Playwright instead of Cypress and, after upgrading to react-router 7, we started too observing a lot of flakiness in browser tests that were pretty stable before. Finding out what the cause was has been really difficult and, after finding this issue and reverting the upgrade, I can confirm too that there is a flaky behaviour during navigation in version 7.
In our case, it seemed like only randomly (much more in CI than locally), some navigation transitions would just not happen and we could not observe any errors in the Playwright tests traces. The tests would fail after 10 seconds as the navigation just didn't seem to happen. It can definitely be noticeable only in browser test frameworks due to the speed at which they click, but it is definitely not a consistent behaviour and a clear showstopper for us to upgrade.

Thanks for the great work and the support!

[keithjgrant]

Our CI has had a lot more flaky tests as well. The timing of navigation has definitely changed, probably due to the use of transitions, at least in part. I've had to knock most if it back by adding assertions to tests to confirm navigation to the new page is complete before proceeding with the test (checking the page title, asserting the clicked tab is now selected, etc).

Asserting the url has changed does NOT work. In my tests, the URL often updates now before the page content does, which was not the case in v6.

[github-actions]

To align with our new Open Governance model, we are now requiring that all issues have a minimal and runnable reproduction. To that end, we're doing some housekeeping in the repo to clean up existing issues that do not have a valid reproduction. This should get us down to a more manageable number of issues and allow us to be more responsive to existing and newly filed issues.

We're using a Github actions script to identify issues without a reproduction by looking for a StackBlitz, CodeSandbox, or Github link in the issue body. This won't be perfect, so if this issue has a reproduction on another platform, please comment back on here and we can re-open the issue. Similarly, if there's a reproduction buried in a comment, please move the link into the description and comment back. Please tag @brophdawg11 or @brookslybrand in your comment so we get a notification as well 🙂.

If this issue did not have a reproduction but is still valid, or if you wish to start with a fresh issue, please create a new issue with a fresh reproduction against v7 and link to this issue in the new description.

If this a feature request, please open a new Proposal Discussion in React Router, and if it gets enough community support it can be considered for implementation.

If you have any questions you can always reach out on Discord. Thanks again for providing feedback and helping us make React Router even better!

[francisco-polaco]

Also running into this. Our Cypress suite started to fail immediately after upgrading React Router to V7. Maybe due to the use of async transitions, Cypress is using old text in the UI when asserting.

Manually I cannot reproduce the issue. Anyone got a workaround?

[pjg]

@francisco-polaco I've downgraded to v6 and this problem went away.

[francisco-polaco]

@francisco-polaco I've downgraded to v6 and this problem went away.

@pjg We're planning on moving to V7 so we can move away from CRA. Any suggestion?

[pjg]

@francisco-polaco producing a reproducible scenario (perhaps with cpu throttling?) is probably the most immediate need to move fixing this issue forward.