Skip to content

Commit 599f19c

Browse files
Configurable verbosity: graph-derived context + delivery-time description materialization (#9)
* Add VerbosityConfiguration to model (from cashapp cashapp#309) Ports cashapp#309's VerbosityConfiguration verbatim into the agnostic model as the config vocabulary for late description assembly. Pure Foundation, builds on the model target. Nothing consumes it yet. * Move localization into the agnostic model Relocates the localized Strings struct, String+Localization, and the {en,de,ru}.lproj Assets from the Parser target into AccessibilitySnapshotModel so late (verbosity-driven) description assembly can run on any platform. - Strings + localized(...) + StringLocalization made public for the Parser. - Model manifests gain defaultLocalization: en + resources: [.process(Assets)]; Parser target drops the Assets resource; Tuist mirrors the move. - Fixes a latent bug: .lproj lookup used subdirectory: Assets, but .process(Assets) flattens .lproj to the bundle root, so de/ru silently fell back to English. Now looks at the bundle root. New StringsLocalizationTests proves de resolves (Taste., not Button.). 31 model tests green; iOS app builds. * Relocate description assembly into the model as a pure verbosity fold Ports the parse-time NSObject.accessibilityDescription(context:) into the model as AccessibilityElement.description(context:verbosity:), reading only stored model data + a graph-derived DerivedContext. This un-bakes the description: it is now a pure, platform-independent function of (element, context, verbosity) that reproduces the historical string at .verbose. - DerivedContext: ref-free mirror of the parser Context; dataTable headers are resolved HeaderText(label,value) rather than live NSObjects, so they stay re-gatable under includesTableContext. - traitPosition (.before/.after/.none) mirrors iOS 18.4 Verbosity > Controls (Speak Before / Speak After / Don't Speak); .after is the historical default. - 11 assembly tests pin the three trait positions, verbosity gating, container context, and hints. 41 model tests green. Parser still uses its own parse-time path; wiring the parser to call this and consuming dataTable cells is the next step. * Emit .series container node for segmented controls (class-free) Step 4a (part 1): make the graph self-describing for series. - Add ContainerType.series to the model (enum + hand-rolled Codable), distinct from .tabBar: series members keep their Button trait and append "N of M", tabs replace Button with "Tab.". - Detect segmented controls class-free via accessibilityContainerType == 11 (private value, read through the public property; same idiom as private trait bits). Confirmed live + specific by accessibility research: steppers/sliders/date-pickers return 0. - The new node is transparent to rendering (flattenToElements drops containers); segment descriptions still come from the parse-time path until the 4c cutover. Byte-identity: testSegmentedControl + testTabBars + full DefaultControls and AccessibilitySnapshotTests suites unchanged (23/23); 41 model tests green. * Classify tab bars class-free via .tabBarItem children Step 4a (part 2): make the graph self-describing for tab bars. A container whose children carry the private .tabBarItem trait (bit 28) is a tab bar. This recognizes a real UITabBar without an `is UITabBar` check: UITabBar reports accessibilityContainerType == .semanticGroup and carries no .tabBar trait, so today it emits as a semanticGroup node; the children-trait rule reclassifies it to .tabBar. Custom .tabBar-trait views remain matched by the trait. The node-type change is transparent to rendering (flattenToElements drops containers) and descriptions still come from the parse-time path, so this is byte-identity-safe until the 4c cutover. Confirmed live byte-identical (bit 28) on iOS 18.5 + 26.3 by accessibility research; UIStepper/UISlider/UIPageControl carry no such children and are not misclassified. Byte-identity: testTabBars + full DefaultControls (incl. page control, stepper, slider) unchanged. SwiftUIListSectionTests failures are pre-existing (fail on baseline without this change). * Derive container context from graph position (model, 4b) Step 4b: pure model-side derivation mirroring the parser's live derivedContext, UIKit-free. AccessibilityContainer.derivedContext(forChildAt:in:) reads a child's context from graph position alone: - .series -> .series(index: ordinal+1, count: siblings) - .tabBar -> .tab(index: ordinal+1, count: siblings) (.tab ==.tabBarItem) - .list -> .listStart / .listEnd (sole child: start only) - .landmark-> .landmarkStart / .landmarkEnd - .dataTable-> resolve stored cells[i] header child-indices into sibling label/value as HeaderText (over the full child set, before pruning) - semanticGroup / scrollable / .none -> nil The {index, count} this computes is the same value VoiceOver derives from _accessibilityRowRange (which for tab bars UIKit itself computes from sibling position) -- so we reproduce it from the tree with no SPI call. Not yet wired into delivery (that is 4c); the parser's parse-time path still produces descriptions. 9 derivation tests; 50 model tests green. * Materialize descriptions at delivery from graph context (4c) Cut the spoken description/hint over from parse-time baking to a delivery-time fold: parse stamps raw facts + container structure, and `materializingDescriptions(verbosity:)` composes the final string from graph-derived context just before on-screen trimming. Wired into both the UIKit and SwiftUI delivery sites; `.verbose` (the default) reproduces historical output. Three latent byte-identity bugs the live cutover surfaced (all dormant while descriptions were baked): - Subview-based list/landmark got context the old parser withheld: emit .list/.landmark only on the container-API path (explicitlyOrdered); subview- vended containers fall back to .semanticGroup, matching superviewContextParent's historical nil. Faithful subview boundaries stay deferred (plan P5). - Doubled hint: the parser baked the composed hint (raw + trait suffix) into element.hint, so re-composition at delivery doubled switch/adjustable hints. buildElement now stores the raw author hint; the suffix is composed once, late. - Important custom content was folded into the description string; the historical parser never did this (the legend renders it separately). Removed. Full snapshot suite: only the pre-existing SwiftUIListSectionTests failures remain (confirmed identical on a clean baseline). 50 model tests + new parity localizer green. * Preserve .list/.landmark node type on the subview path (4c fixup) The 4c cutover briefly remapped subview-vended .list/.landmark containers to .semanticGroup to force description byte-identity. That broke the container structural contract (testListContainerIsAlwaysPreserved / ...Landmark...): a .list/.landmark view must emit its node type regardless of how it vends its children, so query consumers can find it by type. CI Tuist UnitTests caught it. Revert to always emitting .list/.landmark. Subview-based lists now correctly gain their real 'List Start'/'List End' boundaries via graph derivation -- exposing that accessibility information is the point of this refactor, not a divergence to suppress. Both current snapshot fixtures use the container-API path, so reference PNGs are unchanged; 91 UnitTests + full snapshot suite green (only pre-existing TextField/TextView + SwiftUIListSectionTests failures remain). * Contextualize at the render boundary: materializing flatten + container-aware SwiftUI legend (4d) Flattening is the moment the container structure is dropped, so it is now also the moment each element's graph-derived context is folded into its final rendered string. `flattenToElements(verbosity: = .verbose)` derives context per child as it walks (same composition the deleted `materializingDescriptions` pass performed) and returns terminal, render-ready elements. Existing call sites compile unchanged and get materialized descriptions for free; delivery sites flatten the FULL tree (so "X of N" counts and data-table headers derive from complete child sets) and prune the flat array by visibility afterwards. The UIKit render path is unchanged: same flat markers, same per-marker legend, same stored-description reads, byte-identical snapshots. The SwiftUI legend gains an opt-in container-aware mode ported from cashapp#329 (closed): `showContainers` renders the legend hierarchically with dashed container borders and badges. Unlike cashapp#329, the `.element` case composes `description(context:verbosity:)` live from the element's graph position instead of reading the stored string — the graph walk IS the contextualizer on this path, so context is never stored on the element and re-contextualization cannot double (the composer reads only raw facts). - HierarchyColorAssignment: threads DerivedContext down the assign walk, composes at each element, filters by visibility against the full child set; elements keep their flat traversal indices (overlays identical), containers numbered pre-order after all elements - HierarchyLegendView / ContainerLegendEntryView: ported from cashapp#329 - ParsedAccessibilityData.hierarchy: data plumbing so the SwiftUI renderer can reach the graph; the UIKit renderer ignores it - withDescription made public, documented as a terminal projection - ContainerDemo fixture + snapshot tests (with/without containers) Suites: 50 model tests green; UnitTests green; SnapshotTests only pre-existing failures (TextField/TextView, SwiftUIListSection x2); PreviewsTests green except pre-existing locale-dependent testCustomContentDemo (demo formats 2847 via device locale; scheme does not pin a language). * TEMP: record container demo references on the 26.2 CI runner The iOS 26.2 simulator runtime is no longer downloadable from Apple, so the two new container-demo references for the iOS_26 CI matrix entry can't be recorded locally. Gate record mode to 26.2 so the runner records them, and extend the failure-artifact upload to include the Previews reference images (durable improvement) so the recorded PNGs can be harvested. The record-mode flip will be reverted once the images are committed. * Add recorded 26.2 container references; rename HierarchyColorAssignment to ContextualizedHierarchy - Harvest the two ContainerDemo 26.2 reference images recorded on CI (the 26.2 simulator runtime is no longer downloadable locally) and revert the temporary OS-gated recordMode lines. - Rename HierarchyColorAssignment -> ContextualizedHierarchy: the type applies graph-derived context to each element (composed description, hint, and overlay color index), not just color assignment. AssignedNode becomes Node. No behavior change. * Separate marker numbering from ContextualizedHierarchy Numbering (and the color each number selects) is a property of the snapshot rendering, not the hierarchy. ContextualizedHierarchy now carries only context application — description and hint composed from graph position, container structure preserved — and HierarchyLegendView assigns indices as it renders: elements in flat traversal order (matching the markers array), containers pre-order after all elements. No pixel change. * Document ContainerType.tabBar's two capture channels * Model hint decomposition: user fact, computed state utterances, computed spoken merge The user-set hint stays the stored fact (element.hint, raw since the un-bake); the VoiceOver state utterances become a computed accessor (stateHint(verbosity:)); the spoken merge stays the hint half of description(context:verbosity:). All three run the same pipeline — the historical merge rules (switch wraps, text entry replaces, adjustable chains) are factored into one private function that the spoken path feeds the real hint and the state path feeds nil, so spoken == merge(user, state) by construction and nothing new is stored anywhere. Spoken output is byte-identical (verbatim factoring). * One contextualize walk: model owns the canonical→marker projection contextualized(verbosity:) on [AccessibilityHierarchy] is now THE contextualize step — the single walk that threads DerivedContext down the tree and composes each element's spoken strings. flattenToElements becomes contextualize-then-plain-flatten, and the Previews ContextualizedHierarchy.build becomes pure structure (visibility filter + empty-container drop) over the same walk, so both legend modes speak identically by construction and the duplicated context walk is gone. Also documents (finding 149, iOS 26.3): real VoiceOver speaks N-of-M / table position trailing like us, but weaves list/landmark boundary phrases into the trait-specifier position — we keep the historical trailing placement for byte-identity. Gates: 54 model tests; Previews snapshot suite incl. both ContainerDemo refs (sole failure = pre-existing locale-environmental testCustomContentDemo); parity + parser + index-API unit tests green.
1 parent 313e7eb commit 599f19c

37 files changed

Lines changed: 1928 additions & 308 deletions

File tree

.github/workflows/ci.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,4 +97,6 @@ jobs:
9797
if: failure()
9898
with:
9999
name: Reference Images-(${{ matrix.platform }})
100-
path: Example/SnapshotTests/ReferenceImages
100+
path: |
101+
Example/SnapshotTests/ReferenceImages
102+
Example/AccessibilitySnapshotPreviewsTests/ReferenceImages

AccessibilitySnapshotModel/Package.swift

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import PackageDescription
1111
// targets and fails to build on macOS with `'UIKit/UIKit.h' file not found`.
1212
let package = Package(
1313
name: "AccessibilitySnapshotModel",
14+
defaultLocalization: "en",
1415
products: [
1516
.library(
1617
name: "AccessibilitySnapshotModel",
@@ -20,7 +21,8 @@ let package = Package(
2021
targets: [
2122
.target(
2223
name: "AccessibilitySnapshotModel",
23-
path: "Sources/AccessibilitySnapshotModel"
24+
path: "Sources/AccessibilitySnapshotModel",
25+
resources: [.process("Assets")]
2426
),
2527
.testTarget(
2628
name: "AccessibilitySnapshotModelTests",

AccessibilitySnapshotModel/Sources/AccessibilitySnapshotModel/AccessibilityContainer.swift

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,18 @@ public struct AccessibilityContainer: Hashable, Codable, Sendable {
55
case list
66
case landmark
77
case dataTable(rowCount: Int, columnCount: Int, cells: [DataTableCellInfo?])
8+
/// A tab bar. Members announce "Tab. N of M.", with any `Button` trait replaced by "Tab."
9+
///
10+
/// Normalizes UIKit's two channels for tab-ness (the public `.tabBar` trait predates
11+
/// `accessibilityContainerType` by one iOS release, so it's a container role expressed as
12+
/// a trait): custom views union `.tabBar` onto themselves; a real `UITabBar` carries no
13+
/// `.tabBar` trait (it reports `.semanticGroup`) and is recognized by the private
14+
/// `.tabBarItem` trait on its button children instead.
815
case tabBar
16+
/// An ordered series whose members announce a position ("N of M") while keeping their own
17+
/// trait (e.g. a `UISegmentedControl`'s segments render "Segment A. Button. 1 of 3."). Unlike
18+
/// `.tabBar`, the member's `Button` trait is retained rather than replaced with "Tab.".
19+
case series
920
case scrollable(contentSize: AccessibilitySize)
1021
}
1122

@@ -87,6 +98,7 @@ extension AccessibilityContainer.ContainerType {
8798
case landmark
8899
case dataTable
89100
case tabBar
101+
case series
90102
case scrollable
91103
}
92104

@@ -136,6 +148,8 @@ extension AccessibilityContainer.ContainerType {
136148
)
137149
case .tabBar:
138150
self = .tabBar
151+
case .series:
152+
self = .series
139153
case .scrollable:
140154
let nested = try container.nestedContainer(keyedBy: ScrollableKeys.self, forKey: .scrollable)
141155
self = try .scrollable(contentSize: nested.decode(AccessibilitySize.self, forKey: .contentSize))
@@ -162,6 +176,8 @@ extension AccessibilityContainer.ContainerType {
162176
try nested.encode(cells, forKey: .cells)
163177
case .tabBar:
164178
_ = container.nestedContainer(keyedBy: SemanticGroupKeys.self, forKey: .tabBar)
179+
case .series:
180+
_ = container.nestedContainer(keyedBy: SemanticGroupKeys.self, forKey: .series)
165181
case let .scrollable(contentSize):
166182
var nested = container.nestedContainer(keyedBy: ScrollableKeys.self, forKey: .scrollable)
167183
try nested.encode(contentSize, forKey: .contentSize)

AccessibilitySnapshotModel/Sources/AccessibilitySnapshotModel/AccessibilityDelivery.swift

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,12 @@ public extension Array where Element == AccessibilityHierarchy {
3030
/// and any container left with no surviving children dropped (an empty container is not an
3131
/// accessibility element — VoiceOver never stops on it — so it must not remain in the tree).
3232
///
33-
/// This is a tree-to-tree transform. Flatten the result to get the rendered element list:
33+
/// This is a tree-to-tree transform. Note that flattening materializes descriptions from
34+
/// graph-derived context, so for rendering prefer flattening the FULL tree and pruning the flat
35+
/// array by `visibility` — pruning the tree first would derive "X of N" counts and data-table
36+
/// headers from an incomplete child set:
3437
///
35-
/// hierarchy.onscreen().flattenToElements()
38+
/// hierarchy.flattenToElements().filter { $0.visibility == .onscreen }
3639
///
3740
/// The full tree (`self`) still carries the off-screen elements; read `scrollContainerSummaries()`
3841
/// off it — not off the pruned result — to tally what was dropped.

0 commit comments

Comments
 (0)