Skip to content

Conversation

@BenHenning
Copy link
Collaborator

@BenHenning BenHenning commented Aug 6, 2025

The basics

The details

Resolves

Fixes part of #8207
Fixes part of #3370

Proposed Changes

This introduces initial broad ARIA integration in order to enable at least basic screen reader support when using keyboard navigation.

Largely this involves introducing ARIA roles and labels in a bunch of places, sometimes done in a way to override normal built-in behaviors of the accessibility node tree in order to get a richer first-class output for Blockly (such as for blocks and workspaces).

Reason for Changes

ARIA is the fundamental basis for configuring how focusable nodes in Blockly are represented to the user when using a screen reader. As such, all focusable nodes requires labels and roles in order to correctly communicate their contexts.

The specific approach taken in this PR is to simply add labels and roles to all nodes where obvious with some extra work done for WorkspaceSvg and BlockSvg in order to represent blocks as a tree (since that seems to be the best fitting ARIA role per those available: https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Reference/Roles). The custom work specifically for blocks includes:

  • Overriding the role description to be 'block' rather than 'tree item' (which is the default).
  • Overriding the position, level, and number of sibling counts since those are normally determined based on the DOM tree and blocks are not laid out in the tree the same way they are visually or logically (so these computations were incorrect). This is also the reason for a bunch of extra computation logic being introduced.

One note on some of the labels being nonsensical (e.g. 'DoNotOverride?'): this was done intentionally to try and ensure all focusable nodes (that can be focused) have labels, even when the specifics of what that label should be aren't yet clear. More components had these temporary labels until testing revealed how exactly they would behave from a screen reader perspective (at which point their roles and labels were updated as needed). The temporary labels act as an indicator when navigating through the UI, and some of the nodes can't easily be reached (for reasons) and thus may never actually need a label. More work is needed in understanding both what components need labels and what those labels should be, but that will be done beyond this PR.

Test Coverage

No tests are added to this as it's experimental and not a final implementation.

The keyboard navigation tests are failing due to a visibility expansion of connectionCandidate in BlockDragStrategy. There's no way to avoid this breakage, unfortunately. Instead, this PR will be merged and then RaspberryPiFoundation/blockly-keyboard-experimentation#684 will be finalized and merged to fix it. There's some additional work that will happen both in that branch and in a later PR in core Blockly to integrate the two experimentation branches as part of #9283 so that CI passes correctly for both branches.

Documentation

No documentation is needed at this time.

Additional Information

This work is experimental and is meant to serve two purposes:

  • Provide a foundation for testing and iterating the core screen reader experience in Blockly.
  • Provide a reference point for designing a long-term solution that accounts for all requirements collected during user testing.

This code should never be merged into develop as it stands. Instead, it will be redesigned with maintainability, testing, and correctness in mind at a future date (see RaspberryPiFoundation/blockly-keyboard-experimentation#673).

@github-actions github-actions bot added the PR: feature Adds a feature label Aug 6, 2025
@BenHenning
Copy link
Collaborator Author

There might need to be a bit of branch-fu in order to make the tests work correctly between this new long-lived core branch and the equivalent version in the keyboard navigation repo.

@BenHenning
Copy link
Collaborator Author

BenHenning commented Aug 6, 2025

Next steps here:

@BenHenning BenHenning changed the title feat: Add initial support for screen readers feat: Add initial support for screen readers (experimental) Aug 6, 2025
@github-actions github-actions bot added PR: feature Adds a feature and removed PR: feature Adds a feature labels Aug 6, 2025
Comment on lines 221 to 227
aria.setState(svgPath, aria.State.ROLEDESCRIPTION, 'block');
aria.setRole(svgPath, aria.Role.TREEITEM);
this.recomputeAriaLabel();
svgPath.tabIndex = -1;
this.currentConnectionCandidate = null;

this.doInit_();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The aria bit needs to be after doInit_() for the block to have fields.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch. The dataflow of when things happen has changed each time the code was moved, so I missed this and somehow missed the voice output being incomplete for blocks.

@BenHenning BenHenning changed the base branch from add-screen-reader-support to add-screen-reader-support-experimental August 6, 2025 20:28
@github-actions github-actions bot added PR: feature Adds a feature and removed PR: feature Adds a feature labels Aug 6, 2025
@github-actions github-actions bot added PR: feature Adds a feature and removed PR: feature Adds a feature labels Aug 6, 2025
@BenHenning BenHenning marked this pull request as ready for review August 6, 2025 22:05
@BenHenning BenHenning requested a review from a team as a code owner August 6, 2025 22:05
@BenHenning BenHenning requested review from rachel-fenichel and removed request for a team August 6, 2025 22:05
@github-actions github-actions bot added PR: feature Adds a feature and removed PR: feature Adds a feature labels Aug 6, 2025
@github-actions github-actions bot added PR: feature Adds a feature and removed PR: feature Adds a feature labels Aug 6, 2025
@BenHenning BenHenning merged commit d0ad934 into RaspberryPiFoundation:add-screen-reader-support-experimental Aug 6, 2025
13 of 15 checks passed
BenHenning added a commit to RaspberryPiFoundation/blockly-keyboard-experimentation that referenced this pull request Aug 11, 2025
Fixes part of RaspberryPiFoundation/blockly#3370
Fixes part of RaspberryPiFoundation/blockly#9283

This PR introduces the plugin corresponding changes needed by RaspberryPiFoundation/blockly#9280.

In particular, this:
- Introduces CI workflow updates so that "tip-of-tree" is redefined as the corresponding experimental branch in core for screen reader support (to ensure CI passes here for changes that happen there).
- Removes some access error overrides since the corresponding field was made public in the core PR (which is the root of the CI failures here).
- Introduces some general instructions in the navigation plugin playground for folks testing Blockly with a screen reader (and points to #673 for discussion).

**Important**: Note that this PR's changes will NOT be merged into the `main` branch as-is. They will be reconsidered entirely (and likely largely reimplemented) separately from this experimental branch and submitted as a separate, future review for `main`. See RaspberryPiFoundation/blockly#9280 and Fixes part of RaspberryPiFoundation/blockly#9283 for more specifics on why the separate branch thing is being set up this way.
BenHenning added a commit to BenHenning/blockly-keyboard-experimentation that referenced this pull request Aug 11, 2025
* feat: Deploy GH pages w/ screen reader experiment.

* fix: Use correct branch for plugin.

* feat: Introduce initial screen reader support  (experimental) (RaspberryPiFoundation#684)

Fixes part of RaspberryPiFoundation/blockly#3370
Fixes part of RaspberryPiFoundation/blockly#9283

This PR introduces the plugin corresponding changes needed by RaspberryPiFoundation/blockly#9280.

In particular, this:
- Introduces CI workflow updates so that "tip-of-tree" is redefined as the corresponding experimental branch in core for screen reader support (to ensure CI passes here for changes that happen there).
- Removes some access error overrides since the corresponding field was made public in the core PR (which is the root of the CI failures here).
- Introduces some general instructions in the navigation plugin playground for folks testing Blockly with a screen reader (and points to RaspberryPiFoundation#673 for discussion).

**Important**: Note that this PR's changes will NOT be merged into the `main` branch as-is. They will be reconsidered entirely (and likely largely reimplemented) separately from this experimental branch and submitted as a separate, future review for `main`. See RaspberryPiFoundation/blockly#9280 and Fixes part of RaspberryPiFoundation/blockly#9283 for more specifics on why the separate branch thing is being set up this way.
BenHenning added a commit to BenHenning/blockly-keyboard-experimentation that referenced this pull request Aug 11, 2025
* feat: Deploy GH pages w/ screen reader experiment.

* fix: Use correct branch for plugin.

* feat: Introduce initial screen reader support  (experimental) (RaspberryPiFoundation#684)

Fixes part of RaspberryPiFoundation/blockly#3370
Fixes part of RaspberryPiFoundation/blockly#9283

This PR introduces the plugin corresponding changes needed by RaspberryPiFoundation/blockly#9280.

In particular, this:
- Introduces CI workflow updates so that "tip-of-tree" is redefined as the corresponding experimental branch in core for screen reader support (to ensure CI passes here for changes that happen there).
- Removes some access error overrides since the corresponding field was made public in the core PR (which is the root of the CI failures here).
- Introduces some general instructions in the navigation plugin playground for folks testing Blockly with a screen reader (and points to RaspberryPiFoundation#673 for discussion).

**Important**: Note that this PR's changes will NOT be merged into the `main` branch as-is. They will be reconsidered entirely (and likely largely reimplemented) separately from this experimental branch and submitted as a separate, future review for `main`. See RaspberryPiFoundation/blockly#9280 and Fixes part of RaspberryPiFoundation/blockly#9283 for more specifics on why the separate branch thing is being set up this way.
BenHenning added a commit to BenHenning/blockly-keyboard-experimentation that referenced this pull request Aug 11, 2025
* feat: Deploy GH pages w/ screen reader experiment.

* fix: Use correct branch for plugin.

* feat: Introduce initial screen reader support  (experimental) (RaspberryPiFoundation#684)

Fixes part of RaspberryPiFoundation/blockly#3370
Fixes part of RaspberryPiFoundation/blockly#9283

This PR introduces the plugin corresponding changes needed by RaspberryPiFoundation/blockly#9280.

In particular, this:
- Introduces CI workflow updates so that "tip-of-tree" is redefined as the corresponding experimental branch in core for screen reader support (to ensure CI passes here for changes that happen there).
- Removes some access error overrides since the corresponding field was made public in the core PR (which is the root of the CI failures here).
- Introduces some general instructions in the navigation plugin playground for folks testing Blockly with a screen reader (and points to RaspberryPiFoundation#673 for discussion).

**Important**: Note that this PR's changes will NOT be merged into the `main` branch as-is. They will be reconsidered entirely (and likely largely reimplemented) separately from this experimental branch and submitted as a separate, future review for `main`. See RaspberryPiFoundation/blockly#9280 and Fixes part of RaspberryPiFoundation/blockly#9283 for more specifics on why the separate branch thing is being set up this way.

* Revert "feat: Introduce initial screen reader support  (experimental) (RaspberryPiFoundation#684)"

This reverts commit 458e4f8.
BenHenning added a commit to BenHenning/blockly-keyboard-experimentation that referenced this pull request Aug 11, 2025
* feat: Deploy GH pages w/ screen reader experiment.

* fix: Use correct branch for plugin.

* feat: Introduce initial screen reader support  (experimental) (RaspberryPiFoundation#684)

Fixes part of RaspberryPiFoundation/blockly#3370
Fixes part of RaspberryPiFoundation/blockly#9283

This PR introduces the plugin corresponding changes needed by RaspberryPiFoundation/blockly#9280.

In particular, this:
- Introduces CI workflow updates so that "tip-of-tree" is redefined as the corresponding experimental branch in core for screen reader support (to ensure CI passes here for changes that happen there).
- Removes some access error overrides since the corresponding field was made public in the core PR (which is the root of the CI failures here).
- Introduces some general instructions in the navigation plugin playground for folks testing Blockly with a screen reader (and points to RaspberryPiFoundation#673 for discussion).

**Important**: Note that this PR's changes will NOT be merged into the `main` branch as-is. They will be reconsidered entirely (and likely largely reimplemented) separately from this experimental branch and submitted as a separate, future review for `main`. See RaspberryPiFoundation/blockly#9280 and Fixes part of RaspberryPiFoundation/blockly#9283 for more specifics on why the separate branch thing is being set up this way.

* Revert "feat: Introduce initial screen reader support  (experimental) (RaspberryPiFoundation#684)"

This reverts commit 458e4f8.

* chore: Remove unnecessary change dirs.
BenHenning added a commit that referenced this pull request Aug 11, 2025
…(experimental) (#9284)

## The basics

- [x] I [validated my changes](https://developers.google.com/blockly/guides/contribute/core#making_and_verifying_a_change)

## The details
### Resolves

Fixes #9283

### Proposed Changes

Updates the keyboard navigation plugin workflow to point to the screen reader experimentation branch.

### Reason for Changes

This ensures that both the plugin (via RaspberryPiFoundation/blockly-keyboard-experimentation#684) and core Blockly correctly link their experimental screen reader branches against each other to ensure CI passes correctly for both.

### Test Coverage

No new tests are needed here.

### Documentation

N/A

### Additional Information

Similar to #9280, this PR will never be merged directly into `develop` so the workflow changes will remain isolated to the experimental branch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PR: feature Adds a feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants