feat: Introduce initial screen reader support (experimental) #684
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jul 29, 2025
Closed
The branch is no longer dependent on Core changes. Also, expose state that may be needed for experimentation in mover.
This is in the form of a bunch of monkey patches that carefully replace essentially all ARIA initialization happening within core Blockly. This is largely done in a way that won't be well compatible with certain uses of Blockly itself, and is only mean to act as a baseline for testing.
Collaborator
Author
|
This now includes all the active functionality from RaspberryPiFoundation/blockly#9262. |
This also removes some unnecessary code and one TODO.
This property isn't needed right now.
Collaborator
Author
|
Unfortunately, this won't be able to be merged today--I decided to spend the day rearranging the implementation to try and make it a bit better given the concerns multiple folks have surfaced around the difficulty of managing so many monkey patches. I want people to be generally happy with the approach before committing to it since pivoting, while possible, will use up unnecessary time later on if we need to do that. Beyond finalizing the decision, this code breaks existing tests (mostly due to it disabling all of core Blockly's aria logic). This is actually a good thing: it demonstrates one of the key benefits of approaching the experimentation in this way which is quickly seeing how the keyboard navigation behaviors change with aria alterations (though most of these are technically transient failures due to the tests unexpectedly depending on aria labels and roles for non-screen reader cases). |
This adds the missing div needed for ARIA 'announcements'.
This removes unstubbing since it can't really work well, anyway, which fixes the Mocha tests that validate initialization and disposal for the KeyboardNavigation class.
This addresses all lint warnings.
This updates stubbing to use direct function references rather than name strings since the code references will be correctly updated during minification. This also makes it a bit more explicit when non-public functions are being stubbed, and it slightly simplifies stubbing logic.
The decision has been made to set up ARIA directly in core Blockly on a specific long-lived experimental branch.
Collaborator
Author
|
Changes have been removed and moved back into core Blockly: RaspberryPiFoundation/blockly#9280. |
Merged
1 task
BenHenning
changed the title
BenHenning
changed the base branch from
add-screen-reader-support
to
add-screen-reader-support-experimental
BenHenning
added a commit
to RaspberryPiFoundation/blockly
that referenced
this pull request
Aug 6, 2025
## The basics - [x] I [validated my changes](https://developers.google.com/blockly/guides/contribute/core#making_and_verifying_a_change) ## The details ### Resolves Fixes part of #8207 Fixes part of #3370 ### Proposed Changes This introduces initial broad ARIA integration in order to enable at least basic screen reader support when using keyboard navigation. Largely this involves introducing ARIA roles and labels in a bunch of places, sometimes done in a way to override normal built-in behaviors of the accessibility node tree in order to get a richer first-class output for Blockly (such as for blocks and workspaces). ### Reason for Changes ARIA is the fundamental basis for configuring how focusable nodes in Blockly are represented to the user when using a screen reader. As such, all focusable nodes requires labels and roles in order to correctly communicate their contexts. The specific approach taken in this PR is to simply add labels and roles to all nodes where obvious with some extra work done for `WorkspaceSvg` and `BlockSvg` in order to represent blocks as a tree (since that seems to be the best fitting ARIA role per those available: https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Reference/Roles). The custom work specifically for blocks includes: - Overriding the role description to be 'block' rather than 'tree item' (which is the default). - Overriding the position, level, and number of sibling counts since those are normally determined based on the DOM tree and blocks are not laid out in the tree the same way they are visually or logically (so these computations were incorrect). This is also the reason for a bunch of extra computation logic being introduced. One note on some of the labels being nonsensical (e.g. 'DoNotOverride?'): this was done intentionally to try and ensure _all_ focusable nodes (that can be focused) have labels, even when the specifics of what that label should be aren't yet clear. More components had these temporary labels until testing revealed how exactly they would behave from a screen reader perspective (at which point their roles and labels were updated as needed). The temporary labels act as an indicator when navigating through the UI, and some of the nodes can't easily be reached (for reasons) and thus may never actually need a label. More work is needed in understanding both what components need labels and what those labels should be, but that will be done beyond this PR. ### Test Coverage No tests are added to this as it's experimental and not a final implementation. The keyboard navigation tests are failing due to a visibility expansion of `connectionCandidate` in `BlockDragStrategy`. There's no way to avoid this breakage, unfortunately. Instead, this PR will be merged and then RaspberryPiFoundation/blockly-keyboard-experimentation#684 will be finalized and merged to fix it. There's some additional work that will happen both in that branch and in a later PR in core Blockly to integrate the two experimentation branches as part of #9283 so that CI passes correctly for both branches. ### Documentation No documentation is needed at this time. ### Additional Information This work is experimental and is meant to serve two purposes: - Provide a foundation for testing and iterating the core screen reader experience in Blockly. - Provide a reference point for designing a long-term solution that accounts for all requirements collected during user testing. This code should never be merged into `develop` as it stands. Instead, it will be redesigned with maintainability, testing, and correctness in mind at a future date (see RaspberryPiFoundation/blockly-keyboard-experimentation#673).
This was referenced Aug 6, 2025
gonfunko
approved these changes
Aug 11, 2025
BenHenning
merged commit 458e4f8
into
RaspberryPiFoundation:add-screen-reader-support-experimental
8 checks passed
BenHenning
added a commit
to BenHenning/blockly-keyboard-experimentation
that referenced
this pull request
Aug 11, 2025
* feat: Deploy GH pages w/ screen reader experiment. * fix: Use correct branch for plugin. * feat: Introduce initial screen reader support (experimental) (RaspberryPiFoundation#684) Fixes part of RaspberryPiFoundation/blockly#3370 Fixes part of RaspberryPiFoundation/blockly#9283 This PR introduces the plugin corresponding changes needed by RaspberryPiFoundation/blockly#9280. In particular, this: - Introduces CI workflow updates so that "tip-of-tree" is redefined as the corresponding experimental branch in core for screen reader support (to ensure CI passes here for changes that happen there). - Removes some access error overrides since the corresponding field was made public in the core PR (which is the root of the CI failures here). - Introduces some general instructions in the navigation plugin playground for folks testing Blockly with a screen reader (and points to RaspberryPiFoundation#673 for discussion). **Important**: Note that this PR's changes will NOT be merged into the `main` branch as-is. They will be reconsidered entirely (and likely largely reimplemented) separately from this experimental branch and submitted as a separate, future review for `main`. See RaspberryPiFoundation/blockly#9280 and Fixes part of RaspberryPiFoundation/blockly#9283 for more specifics on why the separate branch thing is being set up this way.
BenHenning
added a commit
to BenHenning/blockly-keyboard-experimentation
that referenced
this pull request
Aug 11, 2025
* feat: Deploy GH pages w/ screen reader experiment. * fix: Use correct branch for plugin. * feat: Introduce initial screen reader support (experimental) (RaspberryPiFoundation#684) Fixes part of RaspberryPiFoundation/blockly#3370 Fixes part of RaspberryPiFoundation/blockly#9283 This PR introduces the plugin corresponding changes needed by RaspberryPiFoundation/blockly#9280. In particular, this: - Introduces CI workflow updates so that "tip-of-tree" is redefined as the corresponding experimental branch in core for screen reader support (to ensure CI passes here for changes that happen there). - Removes some access error overrides since the corresponding field was made public in the core PR (which is the root of the CI failures here). - Introduces some general instructions in the navigation plugin playground for folks testing Blockly with a screen reader (and points to RaspberryPiFoundation#673 for discussion). **Important**: Note that this PR's changes will NOT be merged into the `main` branch as-is. They will be reconsidered entirely (and likely largely reimplemented) separately from this experimental branch and submitted as a separate, future review for `main`. See RaspberryPiFoundation/blockly#9280 and Fixes part of RaspberryPiFoundation/blockly#9283 for more specifics on why the separate branch thing is being set up this way.
BenHenning
added a commit
to BenHenning/blockly-keyboard-experimentation
that referenced
this pull request
Aug 11, 2025
BenHenning
added a commit
to BenHenning/blockly-keyboard-experimentation
that referenced
this pull request
Aug 11, 2025
* feat: Deploy GH pages w/ screen reader experiment. * fix: Use correct branch for plugin. * feat: Introduce initial screen reader support (experimental) (RaspberryPiFoundation#684) Fixes part of RaspberryPiFoundation/blockly#3370 Fixes part of RaspberryPiFoundation/blockly#9283 This PR introduces the plugin corresponding changes needed by RaspberryPiFoundation/blockly#9280. In particular, this: - Introduces CI workflow updates so that "tip-of-tree" is redefined as the corresponding experimental branch in core for screen reader support (to ensure CI passes here for changes that happen there). - Removes some access error overrides since the corresponding field was made public in the core PR (which is the root of the CI failures here). - Introduces some general instructions in the navigation plugin playground for folks testing Blockly with a screen reader (and points to RaspberryPiFoundation#673 for discussion). **Important**: Note that this PR's changes will NOT be merged into the `main` branch as-is. They will be reconsidered entirely (and likely largely reimplemented) separately from this experimental branch and submitted as a separate, future review for `main`. See RaspberryPiFoundation/blockly#9280 and Fixes part of RaspberryPiFoundation/blockly#9283 for more specifics on why the separate branch thing is being set up this way. * Revert "feat: Introduce initial screen reader support (experimental) (RaspberryPiFoundation#684)" This reverts commit 458e4f8.
BenHenning
added a commit
to BenHenning/blockly-keyboard-experimentation
that referenced
this pull request
Aug 11, 2025
* feat: Deploy GH pages w/ screen reader experiment. * fix: Use correct branch for plugin. * feat: Introduce initial screen reader support (experimental) (RaspberryPiFoundation#684) Fixes part of RaspberryPiFoundation/blockly#3370 Fixes part of RaspberryPiFoundation/blockly#9283 This PR introduces the plugin corresponding changes needed by RaspberryPiFoundation/blockly#9280. In particular, this: - Introduces CI workflow updates so that "tip-of-tree" is redefined as the corresponding experimental branch in core for screen reader support (to ensure CI passes here for changes that happen there). - Removes some access error overrides since the corresponding field was made public in the core PR (which is the root of the CI failures here). - Introduces some general instructions in the navigation plugin playground for folks testing Blockly with a screen reader (and points to RaspberryPiFoundation#673 for discussion). **Important**: Note that this PR's changes will NOT be merged into the `main` branch as-is. They will be reconsidered entirely (and likely largely reimplemented) separately from this experimental branch and submitted as a separate, future review for `main`. See RaspberryPiFoundation/blockly#9280 and Fixes part of RaspberryPiFoundation/blockly#9283 for more specifics on why the separate branch thing is being set up this way. * Revert "feat: Introduce initial screen reader support (experimental) (RaspberryPiFoundation#684)" This reverts commit 458e4f8. * chore: Remove unnecessary change dirs.
BenHenning
added a commit
to RaspberryPiFoundation/blockly
that referenced
this pull request
Aug 11, 2025
…(experimental) (#9284) ## The basics - [x] I [validated my changes](https://developers.google.com/blockly/guides/contribute/core#making_and_verifying_a_change) ## The details ### Resolves Fixes #9283 ### Proposed Changes Updates the keyboard navigation plugin workflow to point to the screen reader experimentation branch. ### Reason for Changes This ensures that both the plugin (via RaspberryPiFoundation/blockly-keyboard-experimentation#684) and core Blockly correctly link their experimental screen reader branches against each other to ensure CI passes correctly for both. ### Test Coverage No new tests are needed here. ### Documentation N/A ### Additional Information Similar to #9280, this PR will never be merged directly into `develop` so the workflow changes will remain isolated to the experimental branch.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes part of RaspberryPiFoundation/blockly#3370
Fixes part of RaspberryPiFoundation/blockly#9283
This PR introduces the plugin corresponding changes needed by RaspberryPiFoundation/blockly#9280.
In particular, this:
Important: Note that this PR's changes will NOT be merged into the
mainbranch as-is. They will be reconsidered entirely (and likely largely reimplemented) separately from this experimental branch and submitted as a separate, future review formain. See RaspberryPiFoundation/blockly#9280 and Fixes part of RaspberryPiFoundation/blockly#9283 for more specifics on why the separate branch thing is being set up this way.