Add container visualization for accessibility snapshots#278
Draft
RoyalPineapple wants to merge 8 commits intomainfrom
Draft
Add container visualization for accessibility snapshots#278RoyalPineapple wants to merge 8 commits intomainfrom
RoyalPineapple wants to merge 8 commits intomainfrom
Conversation
ab737b5 to
b25c3f7
Compare
3d8c515 to
3733a88
Compare
452d709 to
b531faf
Compare
22e4ff1 to
5b522ad
Compare
5102b9b to
7a2587e
Compare
f887155 to
dc8efaa
Compare
RoyalPineapple
commented
Jan 27, 2026
e2c65ec to
bb6609f
Compare
bb6609f to
2635b9f
Compare
3 tasks
2635b9f to
2809649
Compare
5 tasks
bacc4d0 to
a0352bf
Compare
2cbf24f to
c3a6932
Compare
When showContainers is enabled in the configuration, the legend renders a hierarchical view that groups elements by their accessibility containers using dashed borders and container badges. Scope: legend-only. All element overlay rendering (ElementOverlay, positions, colors, coordinates) is completely unchanged. The snapshot image and its overlays are rendered identically regardless of showContainers — only the legend content differs. New files: - HierarchyColorAssignment: assigns color indices to hierarchy nodes (elements use traversal-order position to match overlay colors; containers use a separate counter) - ContainerLegendEntryView: dashed-border legend entry with a badge that displays the container type (Semantic Group/List/Landmark/ Data Table/Tab Bar) or its label/value/identifier - HierarchyLegendView: recursive hierarchical legend renderer - ContainerDemo: demo using SemanticGroupWrapper to create real UIKit accessibility containers from a SwiftUI demo (SwiftUI has no native equivalent for UIAccessibilityContainerType.semanticGroup) Changes: - AccessibilitySnapshotConfiguration: add showContainers flag - ParsedAccessibilityData: add hierarchy tree alongside flat markers - PreParsedAccessibilitySnapshotView: swap legend based on showContainers; snapshot+overlay section untouched - SwiftUIAccessibilitySnapshotContainerView: pass hierarchy through Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Composites the snapshot image with its element overlays into a single UIImage inside PreParsedAccessibilitySnapshotView, so overlay coordinates are locked to the snapshot pixels before the legend participates in the surrounding SwiftUI layout. This guarantees a 1:1 mapping between the snapshot and its overlays regardless of legend placement. Also swaps parse/render order in AccessibilitySnapshotBaseView so marker frames and the rendered image are taken from the same pre-render layout state. drawHierarchy(afterScreenUpdates: true) mutates layout for nested UIHostingControllers inside UIViewRepresentable wrappers (e.g. the SemanticGroupWrapper demo), which previously caused the first group's overlay to shift relative to the captured snapshot. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Splits the single ContainerDemo into a focused set of demo views that each exercise one container shape the parser produces: - ContainerDemo (updated) — four semanticGroup info variants: label only, label + value, identifier only, and label + value + id - ListContainerDemo — accessibilityContainerType = .list - LandmarkContainerDemo — accessibilityContainerType = .landmark - TabBarContainerDemo — accessibilityTraits = .tabBar on a non-UITabBar view - DataTableContainerDemo — accessibilityContainerType = .dataTable with real UIAccessibilityContainerDataTableCell elements reporting per-cell row and column ranges, so the legend shows "Data Table (r × c)" and each cell's row/column context rather than NSNotFound noise Shared AccessibilityContainers.swift provides: - A generic UIViewRepresentable that wraps SwiftUI content in a UIView reporting a configured UIAccessibilityContainerType or the .tabBar trait - An AccessibilityDataTable UIViewRepresentable that renders a 2D grid of labels and exposes each cell as a real UIAccessibilityElement conforming to UIAccessibilityContainerDataTableCell One test per new demo (all with showContainers: true) plus iOS 26.4 reference images. testContainerDemo/testContainerDemoWithoutContainers references re-recorded for the new semanticGroup variants layout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SwiftUITableDemo exercises native SwiftUI Table accessibility on iPhone: the parser does see accessibilityContainerType = .dataTable, but Table collapses to a one-column List on iPhone and reports rowCount = 0, columnCount = 0 with no UIAccessibilityContainerDataTableCell elements, so the legend shows "Data Table (0 × 0)". Container legend entries now expand to the full available width (maxWidth: .infinity) so badges like "Data Table (0 × 0)" sit on a single line instead of wrapping to fit the narrowest child. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8d0a441 to
b7aa512
Compare
The outer hostingController.safeAreaRegions = [] flipped the rendering of every SwiftUI snapshot in the project, breaking testSimpleView and both SwiftUIListSectionTests across iOS 17/18/26 because their reference images were recorded with the safe area inset on. The PreParsedAccessibilitySnapshotView bake step has its own internal safeAreaRegions = [] for compositing, so removing the outer one has no effect on the new container tests' correctness — only their bytes shift slightly, hence the rerecord.
The previous fix removed `safeAreaRegions = []` from every SwiftUI snapshot, breaking the AccessibilitySnapshotPreviewsTests which expect edge-to-edge rendering through the new bake pipeline. Scope the opt-out to the SwiftUI layout engine so the legacy UIKit engine keeps the safe area inset its references were recorded with. Re-record the affected references: - iOS 18.5: all 13 SwiftUIRendererTests now match the bake-step output - iOS 26.4: all 13 SwiftUIRendererTests, including refs that didn't exist before for the 6 pre-existing tests - iOS 17.5: testTabBars rebaselined for sub-pixel drift introduced by the parse-before-render reorder
Sub-pixel rendering differs between local Macs and the GitHub Actions runners, so references recorded locally don't match what CI produces. Replace the affected references with the actual captured images from the previous CI run's Failed Image attachments. For the 7 new container tests on iOS 26.2, seed placeholders (copies of the 26.4 references) so CI actually renders them on the next run instead of bailing with "Reference image not found"; the resulting Failed Images can replace these placeholders.
The previous commit seeded placeholders (copies of the iOS 26.4 references) for 7 tests so CI would render them and emit Failed Image attachments. This commit swaps those in for the actual iOS 26.2 captures from CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds container visualization to accessibility snapshot legends without changing element overlay behavior.
Summary
showContainersconfiguration flag turns on a hierarchical legend that groups elements by their accessibility container with dashed borders and per-container badgesUIImagebefore the legend enters layout, so overlay positions can never be shifted by legend placementUIHostingControllercontent (e.g.SemanticGroupWrapper)TablerevealDetails
Snapshot + overlay baking
PreParsedAccessibilitySnapshotView.initcomposites the snapshot image and all element overlays into a single flatUIImageup front. The renderedsnapshotWithOverlaysbody is justImage(uiImage:)— noZStackof overlays on top of the image. This decouples overlay alignment from the surrounding SwiftUI layout, so the legend (right, below, multi-column, future flow layouts) can never shift overlay coordinates.Baking uses
drawHierarchy(afterScreenUpdates: true)inside a temporaryUIWindowwithsafeAreaRegions = []so SwiftUI content renders correctly without safe-area offsets.Overlay alignment fix
drawHierarchy(afterScreenUpdates: true)mutates layout mid-render for views that defer layout until the render cycle — notably nestedUIHostingControllerinstances insideUIViewRepresentablewrappers (e.g.SemanticGroupWrapper). When parsing happened after rendering, the parser readaccessibilityFramevalues from a post-render layout that no longer matched what the image captured, shifting the first group's overlay.Fix: swap the parse/render order in
AccessibilitySnapshotBaseView.parseAccessibility()so marker frames and the rendered image both come from the same pre-render layout state.Container legend
When
showContainersis enabled, the legend renders a hierarchical view that groups elements by their accessibility containers using dashed borders and container badges. Container entries expand to the full legend column width so multi-word badges like "Data Table (3 × 4)" stay on one line. The snapshot area is identical regardless ofshowContainers— only the legend content changes.Supported container types in the badge label:
semanticGroup(label, value, identifier)→ "label: value", or identifier, or "Semantic Group"list→ "List"landmark→ "Landmark"dataTable(rows, cols)→ "Data Table (r × c)"tabBar→ "Tab Bar"New types:
HierarchyColorAssignment— assigns color indices to hierarchy nodes (elements use traversal-order position matching overlay indices; containers use a separate counter)ContainerLegendEntryView/ContainerBadge— dashed-border legend entry with layered-icon badgeHierarchyLegendView— recursive hierarchical legend rendererDemos and tests
Each container type now has a focused demo + test under
AccessibilitySnapshotPreviewsDemo:ContainerDemo— foursemanticGroupinfo variants (label only, label + value, identifier only, all three)ListContainerDemo—accessibilityContainerType = .listLandmarkContainerDemo—accessibilityContainerType = .landmarkTabBarContainerDemo—accessibilityTraits = .tabBaron a non-UITabBarviewDataTableContainerDemo—accessibilityContainerType = .dataTablewith realUIAccessibilityContainerDataTableCellelements reporting per-cell row/column ranges, so the legend shows "Data Table (3 × 4)" plus each cell's row/column contextSwiftUITableDemo— exposes how SwiftUI's nativeTablecollapses to a one-column List on iPhone; the parser still emits a.dataTablecontainer but withrowCount = 0, columnCount = 0and noUIAccessibilityContainerDataTableCellelementsShared
AccessibilityContainers.swiftprovides:UIViewRepresentablethat wraps SwiftUI content in aUIViewreporting a configuredUIAccessibilityContainerTypeor the.tabBartraitAccessibilityDataTableUIViewRepresentablethat renders a 2D grid of labels and exposes each cell as a realUIAccessibilityElementconforming toUIAccessibilityContainerDataTableCellOther changes
FBSnapshotTestCase+SwiftUI: setshostingController.safeAreaRegions = []on iOS 16.4+ in the config-based overload, preventingdrawHierarchyfrom baking status bar / safe-area offsets into the snapshotAccessibilitySnapshotConfiguration: newshowContainers: Bool = falseflagParsedAccessibilityData: now also carries the fullhierarchy: [AccessibilityHierarchy]tree (not just flattened markers) so the container legend can renderTest plan
🤖 Generated with Claude Code