chore: Carousel or Image Rotator RFC

[nwidynski]

View Rendered RFC

Related discussions:

• useTab() does not adhere to W3C ARIA APG regarding aria-controls #8699
• Feat: Add lifecycle callback to useId #8630
• Feat: Add support for multiple adjacent collections #8553

✅ Pull Request Checklist:

• Included link to corresponding React Spectrum GitHub Issue.
• Added/updated unit tests and storybook for this change (for new code or code which already has tests).
• Filled out test instructions.
• Updated documentation (if it already exists for this component).
• Looked at the Accessibility Practices for this feature - Aria Practices

📝 Test Instructions:

🧢 Your Project:

[LFDanLu]

Thanks for all your hard work on the RFC! The team was able to take a look at it briefly today (and will hopefully carve out some more time to read through it again and discuss more deeply) and the preliminary feedback/gut feelings is as follows:

• Patterns like Tabs and GridList/ListBox are featured heavily, perhaps we can extend those hooks instead of making a separate carousel package?
  • However, from my reading of this RFC, there are considerations to be made when binding the various extra controls to the carousel's elements as well as considerations for announcements for when elements are scrolled into/out of view for presentational carousels
• Have you been able to or tried to build this Carousel yourself already with the existing hooks/components (Tabs/GridList/CardView/etc)? I think from previous conversations in related issue point to yes (especially since you've highlighted difficulties with getting infinite looping to work), but its a bit hard to tell across the disparate conversations haha. I think it would be helpful if you could add some of the pain points/related discussions as supplements for why some of the additional hooks/delegates/etc should be created if possible, if only to make it less theoretical sounding if that makes sense.
• If we were to only support "Tabbed" for now, what could be dropped for a first pass? Adding this incrementally would be ideal and easier to digest for the team

[LFDanLu]

something that isn't quite clear to me is whether or not a "presentational" carousel is truly a carousel or just a horizontally scrolling cardview/listbox/gridlist. I suppose this is where the screenreader announcements and support for the various carousel controls may play a role in differentiating them, but I'll need to check the aria pattern.

[nwidynski]

Hopefully answered by the other comments, but TLDR; A presentational carousel is basically just a useSelectableCollection which has given up control over its scroll offset to ScrollManager.

ARIA semantics of the controlled collection are largely left untouched, depending on how we decide to model the relationship between picker dot and controlled item/scroll target. As you already suspected, the essence of a carousel lies in the ARIA semantics of controls and their announcement of scroll/visibility state.

[LFDanLu]

similar to as mentioned above, these experiences feel like behaviors covered in part by existing components (tabs/listbox/gridlist/etc) so I wonder if they can be extensions of those components somehow.

A RAC level Carousel component seems reasonable, but maybe we could extend the existing hooks for those patterns rather than create new carousel hooks?

[nwidynski]

Just to be sure, are you referencing carousel hooks or rotator hooks? The following TLDR; will hopefully make it easier to reason about their distinct scopes:

• @react-aria/rotator -> A more advanced ScrollView, with built-in scrollIntoView & a playback delegate to answer "which scroll target comes next when i press the next button?"

• @react-aria/carousel -> W3C spec, event wiring for playback controls & a live announcer for the active target.

Now I don’t think we would want to put an entirely different W3C spec inside of another package, so I figure you are referring to the rotation part? In any scenario,the existing listbox and gridlist hooks continue to be run next to the ones we add here, and we do rely on them. Hopefully I can make this clearer by providing the hook representation for a virtualized presentational carousel:

<useCarousel & useListState & useScrollManagerState>              // => <Carousel />
    <useButton slot="previous" />
    <useButton slot="next" />

    <useSlidePickerList>                                          // => <SlidePickerList /> 
        <useSlidePicker id="1" />
        <useSlidePicker id="2" />
        <useSlidePicker id="3" />
    </>

    <useVirtualizerState>                                         // => <Virtualizer />
        <useSelectableCollection & useListState & useScrollState> // => <ListBox />
            <useSelectableItem & useScrollTarget id="1" />
            <useSelectableItem & useScrollTarget id="2" />
            <useSelectableItem & useScrollTarget id="3" />
        </>
    </>
</>

The only reason for why useTab can not also be re-used here for pickers is because it's not really built for virtualization of panels, enforces a specific JSX placement and requires a tab list state instead of a list state.

I could only see this integrate even deeper if the @react-aria/selection package were to internalize the @react-aria/rotator package, replacing the functionality of scrollIntoView. I purposefully decided not to do that, because I felt like the selection package is already complicated enough as is.

For further clarification, please also reference these sections in the RFC (copy & paste the links, click doesnt work for highlight)

• Migration path in implementation strategy (state)

• Migrating path between animation model and implementation strategy (ARIA)

• Focus management in presentational carousels

[LFDanLu]

gotcha, I guess would the Carousel provide some/all of the carousel item props via context or would there be a useCarouselItem hook here? My initial thought would be that we could consider adding carousel specific behaviors/aria attributes into the listbox/gridview hooks if those behaviors/aria attributes were "lightweight" per say, but that doesn't seem to be the case

[nwidynski]

The W3C Spec for the Carousel Pattern has surprisingly little requirements for carousel items, aka. Slide elements. In fact, the only requirement is for them to carry a valid ARIA role and unique accessible name. Since we can guarantuee listbox/gridview items to already meet this criteria, the only case where useSlide would likely actually be leveraged is in a tabbed carousel w/ SlideShow.

Otherwise, there is useScrollTarget which is coming from the @react-aria/rotator package and is used to report scroll area data into its ScrollDelegate. Similarly to useVirtualizerItem, this hook is invoked for each item via a collection renderer - for which #8523 (comment) is the blocker.

In case there are certain accessibility properties i missed out on during exploration, they could probably be internalized within the listbox/gridview hooks like you mentioned 👍

[LFDanLu]

So these below are all mainly to support snap scrolling for a virtualized experience correct?

[nwidynski]

Primarily yes, although they also take on a couple other major responsibilities:

• determining when rotation control state is updated (eager or lazy)
• calculating the active scroll target(s) based on layout and collected ScrollArea data
• determining the playback order of targets when controls are engaged, aka. "which item do we scroll to when hitting next/previous"
• allowing for easy extension/customization, similar to different layouts

The classes are designed to be very computationally light, since they do not stay in sync with layout, but rather just call the layout for the current state when rotation is scheduled or observed. Extension capabilities are especially useful for a component to customize its own rotation behavior, e.g. for when it wants certain items to be skipped w/o the user having to exclude them via #9084 (comment).

For an alternative, I also considered pushing the active target into persisted keys before performing the scroll, but the information of just the target element is often times not enough to reliably scroll to a "stable" position, when snap properties are not uniformly applied. It's better to have items self report scroll & snap properties, similar to how we update their actual size in a virtualizer.

Without these classes, all remaining responsibilities would still have to be managed somehow, which I think we would always do in the state layer thereby requiring some form of abstraction from the DOM. This insight made scroll observation largely appear similar to virtualizer to me, hence why i modeled after it. One determines which items are in view, the other determines which of the items in view are currently active.

[LFDanLu]

Just to clarify, this means the scroll delegate will essentially report the section and its first/last child as the current "in view" items in the context of a carosel?

[nwidynski]

Yes, although rather "active" than "in view". "Active" meaning the picker controls would show their dots as selected, if present in the picker collection.

The display of multiple active targets can also be approached in different ways. Either we carry a multi select list state for rotation controls or only report the "deepest" collection item as active and then simply render parents as selected if a child is active. I think a multi select list state kind of makes more sense, because ScrollDelegate could theoretically report a different active target for each axis.

[LFDanLu]

I'm not quite sure I understand the nuance between the available targets versus the items of the controlled collection here, mind expanding on this?

[nwidynski]

With the RAC API proposed here, we got 2 collections being hoisted up and synchronized:

• the controlled collection (aka. listbox or slideshow)
• the picker control collection.

The later is being used to filter the controlled, which results in our "available targets" collection (if picker is not present, the entire controlled collection is forwarded).

This "available targets" collection is what constructs the ListState in the outer Carousel, which is passed as the delegate to ScrollManager, resulting in all rotation controls only acting on these targets.

Selecting a subset of items for rotation controls can simply be done by rendering certain ids only in the picker. The next/previous buttons will be disabled as soon as the last/first available target is determined as the active target by the ScrollDelegate. This is super helpful when only wanting to rotate on either sections or items for example, or when wanting to skip certain items in rotation (e.g section headers or disabled items).

It also solves a common problem in multi-view carousels, in which we might have targets that can never be scrolled to (because min/max scroll offset is already reached). Since we wouldn't want these items to appear as available dots, we can simply measure the items, determine which ones can never be reached, and remove them from the available targets collection.

[LFDanLu]

I see, thanks for the explanation. Having the picker collection filter the listbox/slideshow is interesting and certainly gives the user a great deal of flexibility in how they'd like the slide show controls to work and what slides they'd want to actually expose to the user regardless of what the listbox/slideshow actually contains.

However, in practice would these primarily always be section headers/disabled items like you mentioned? If so, would we need this degree of flexibility or could be have a single collection and have those section header/disabled item nodes contain information within themselves that the picker could then use to create a filtered subcollection? I'm primarily concerned about the possible complexity with synchronizing those 2 collections.

It also solves a common problem in multi-view carousels, in which we might have targets that can never be scrolled to (because min/max scroll offset is already reached)

I'm not 100% sure what this means, does this imply that there might be additional slides beyond the "last" one offered by the picker that the user shouldn't be able to get to?

[nwidynski]

However, in practice would these primarily always be section headers/disabled items like you mentioned?

Certain node types could definitely self-opt-out from being included in the picker collection. I'm thinking custom node types, loaders, section headers, and basically anything that can be statically analyzed. Where I'm thinking this could be difficult is with a more dynamic state that doesn't live on the node type, as is the case with disabled.

Additionally, a user may select to show either only items, or only sections, which may make it difficult to filter statically, since both node types are valid candidates for certain use cases.

If so, would we need this degree of flexibility

Furthermore to the use cases above, a user may want to configure his multi-view carousel to scroll "one page" at a time. Having very granular control of picker elements, means we can dynamically measure how many items are in a page and then offer one dot per "visual" page - even when items are of different sizes.

Maybe we can find a way to cut down on the exposed flexibility, while remaining with granular control internally though. I will put some thought into that 👍

does this imply that there might be additional slides beyond the "last" one offered by the picker that the user shouldn't be able to get to?

Imagine a multi-view carousel (e.g. 3 items in visible rect), which is scrolled to 100%. When determining the active target based on start alignment, the active target will always be the item at n-2, while item n-1 and n can never be active. Showing these items as picker dots may only make sense if there is further scroll offset after latest item (e.g. scroll-padding) so that they could become the active.

This flexibility would allow us to dynamically measure and remove these items from the picker collection.

I'm primarily concerned about the possible complexity with synchronizing those 2 collections.

Maybe my wording in the comment above was unfortunate. It is of course only one collection being hoisted up by CollectionBuilder, which is then split into the 2 required collections based on the internal prefix described in the RFC. This has worked pretty straightforward so far in the exploration, but let me know if you or the team got any specific concerns.

[LFDanLu]

Interesting, our swippable tabs example has this shouldForceMount applied on each TabPanel individually via a dynamic renderer, but I suppose a API like this enforces the all or nothing distinction you make below.

[nwidynski]

Yep, exactly. I figured that a differentiation on an individual basis would not really make sense in a carousel, although its definitely not exclusive.

The API design effectively forces you to use SlideShow for when you want a tabbed carousel, regardless of whether or not you (force)mount into a scroll container. Only when you want a presentational carousel is when you switch to a ListBox or GridList.

[LFDanLu]

just to clarify, is this describing infinite scroll + wrapping where hitting the end of your data will loop back to the first slide? If yes, then I'm not quite sure the difference between this and "hierarchical".

[LFDanLu]

Oh never mind, I think I understand from the diagram below.

• unordered => it always gives the impression of a loop
• hierachical => it initially appears to be like a list (defined start) but then transitions to the infinite loop upon hitting the last bit of data

[nwidynski]

Yep :) Although it doesn't have to be the last bit of data. Netflix for example does this in their "Top 10" display, where they want you to explore at least one page before transitioning to unordered infinite.

My ideal way to implement this would be with a sentinel, but that may be challenging to place in JSX.

[nwidynski]

@LFDanLu Thank you and the entire team so much for carving out some time, especially when extra low on spare cycles!

I hope my answers cover the reasons as to why these extra delegates are necessary, but feel free to reach out again once you've given it another pass. To address the two remaining questions of how a minimal implementation could look like and how much I implemented of this already:

As you may be able to imagine, writing this detailed of an RFC required me to explore implementation already, so yes - i've given pretty much all of what is proposed here a rough pass already. These explorations have produced the following PR's for the roadblocks I've encountered. I've started to split these out and have polished them, as they should be universally applicable, regardless of this RFC:

• #8533
• #9146
• #9115
• #8523

Additionally, I've singled out some of the difficulties of implementing this RFC in the "Backwards compatibility" section, of which the DOMLayoutDelegate problem is by far the most difficult, because of the way the dependencies are structured (my hope is this will be rendered easier with the dependency RFC).

In general, what I've figured from my first implementation pass is that this RFC does not really require modifications to the existing codebase and can largely built on top of what is there today, given a few distinct adjustments. What guided the RFC here is rather my take on solving the following requirements:

• Support for multi-view, interactive card carousels
• Support for virtualization and async loading
• Support for infinite looping
• Must have low impact on bundle size
• Must be accessible

Prior to this RFC, we've basically implemented our way through 2 iterations of a carousel, which you've picked up on in some of the things I've shared.

The first one was from scratch using a custom fork of useSelectableCollection, which has prompted the discovery of the event leaks. This solution duplicated a lot of code and would've been hard to maintain. It also only supported the presentational style, while we've had design constraints to support both styles.

The second was based on the tabs pattern, but expanded for infinite looping via fake DOM elements inserted through a custom renderer, which caused focus problems. It also had issues with ARIA semantics, especially in virtualized scenarios, effectively only supporting tabbed style.

The third iteration is now based on virtualizer and layout, which was the groundwork to this RFC. It is written to sit on top of the entire existing collection logic and was designed to be very easy to up/downgrade, while supporting all constraints and use-cases. All ARIA semantics of gridlist and listbox as well as their focus restoration is maintained, thereby not prompting any new accessibility patterns besides for controls.

A migration between cross fading and a scroll container is effectively just:

Carousel.tsx (pseudo-code, tabbed)

let state = useListState();

<button slot="next" onClick={() => state.selectNext()} />

Carousel.tsx (pseudo-code, tabbed, scroll container)

let state = useListState();
let scrollManager = useScrollManagerState({ delegate: state.selectionManager });

<button slot="next" onClick={() => scrollManager.scrollNext()} />

To answer which parts could be dropped when only supporting tabbed, I need to know whether that shall include support for mounting into a scroll container or not. As long as scroll containers shall be supported, then it doesn't make much of a difference as far as the core implementation goes, because the entire @react-aria/rotator package would have to be implemented either way.

If that‘s not the case, then we can skip the rotator package for a first pass. Of course features like infinite mode could always be dropped also. As part of this RFC, I already scoped out a rough roadmap, which you find at the bottom.

[nwidynski]

Just to be sure, are you referencing carousel hooks or rotator hooks? The following TLDR; will hopefully make it easier to reason about their distinct scopes:

• @react-aria/rotator -> A more advanced ScrollView, with built-in scrollIntoView & a playback delegate to answer "which scroll target comes next when i press the next button?"

• @react-aria/carousel -> W3C spec, event wiring for playback controls & a live announcer for the active target.

Now I don’t think we would want to put an entirely different W3C spec inside of another package, so I figure you are referring to the rotation part? In any scenario,the existing listbox and gridlist hooks continue to be run next to the ones we add here, and we do rely on them. Hopefully I can make this clearer by providing the hook representation for a virtualized presentational carousel:

<useCarousel & useListState & useScrollManagerState>              // => <Carousel />
    <useButton slot="previous" />
    <useButton slot="next" />

    <useSlidePickerList>                                          // => <SlidePickerList /> 
        <useSlidePicker id="1" />
        <useSlidePicker id="2" />
        <useSlidePicker id="3" />
    </>

    <useVirtualizerState>                                         // => <Virtualizer />
        <useSelectableCollection & useListState & useScrollState> // => <ListBox />
            <useSelectableItem & useScrollTarget id="1" />
            <useSelectableItem & useScrollTarget id="2" />
            <useSelectableItem & useScrollTarget id="3" />
        </>
    </>
</>

The only reason for why useTab can not also be re-used here for pickers is because it's not really built for virtualization of panels, enforces a specific JSX placement and requires a tab list state instead of a list state.

I could only see this integrate even deeper if the @react-aria/selection package were to internalize the @react-aria/rotator package, replacing the functionality of scrollIntoView. I purposefully decided not to do that, because I felt like the selection package is already complicated enough as is.

For further clarification, please also reference these sections in the RFC (copy & paste the links, click doesnt work for highlight)

• Migration path in implementation strategy (state)

• Migrating path between animation model and implementation strategy (ARIA)

• Focus management in presentational carousels

[nwidynski]

Primarily yes, although they also take on a couple other major responsibilities:

• determining when rotation control state is updated (eager or lazy)
• calculating the active scroll target(s) based on layout and collected ScrollArea data
• determining the playback order of targets when controls are engaged, aka. "which item do we scroll to when hitting next/previous"
• allowing for easy extension/customization, similar to different layouts

The classes are designed to be very computationally light, since they do not stay in sync with layout, but rather just call the layout for the current state when rotation is scheduled or observed. Extension capabilities are especially useful for a component to customize its own rotation behavior, e.g. for when it wants certain items to be skipped w/o the user having to exclude them via #9084 (comment).

For an alternative, I also considered pushing the active target into persisted keys before performing the scroll, but the information of just the target element is often times not enough to reliably scroll to a "stable" position, when snap properties are not uniformly applied. It's better to have items self report scroll & snap properties, similar to how we update their actual size in a virtualizer.

Without these classes, all remaining responsibilities would still have to be managed somehow, which I think we would always do in the state layer thereby requiring some form of abstraction from the DOM. This insight made scroll observation largely appear similar to virtualizer to me, hence why i modeled after it. One determines which items are in view, the other determines which of the items in view are currently active.

[nwidynski]

Yes, although rather "active" than "in view". "Active" meaning the picker controls would show their dots as selected, if present in the picker collection.

The display of multiple active targets can also be approached in different ways. Either we carry a multi select list state for rotation controls or only report the "deepest" collection item as active and then simply render parents as selected if a child is active. I think a multi select list state kind of makes more sense, because ScrollDelegate could theoretically report a different active target for each axis.

[nwidynski]

With the RAC API proposed here, we got 2 collections being hoisted up and synchronized:

• the controlled collection (aka. listbox or slideshow)
• the picker control collection.

The later is being used to filter the controlled, which results in our "available targets" collection (if picker is not present, the entire controlled collection is forwarded).

This "available targets" collection is what constructs the ListState in the outer Carousel, which is passed as the delegate to ScrollManager, resulting in all rotation controls only acting on these targets.

Selecting a subset of items for rotation controls can simply be done by rendering certain ids only in the picker. The next/previous buttons will be disabled as soon as the last/first available target is determined as the active target by the ScrollDelegate. This is super helpful when only wanting to rotate on either sections or items for example, or when wanting to skip certain items in rotation (e.g section headers or disabled items).

It also solves a common problem in multi-view carousels, in which we might have targets that can never be scrolled to (because min/max scroll offset is already reached). Since we wouldn't want these items to appear as available dots, we can simply measure the items, determine which ones can never be reached, and remove them from the available targets collection.

[nwidynski]

Yep, exactly. I figured that a differentiation on an individual basis would not really make sense in a carousel, although its definitely not exclusive.

The API design effectively forces you to use SlideShow for when you want a tabbed carousel, regardless of whether or not you (force)mount into a scroll container. Only when you want a presentational carousel is when you switch to a ListBox or GridList.

[nwidynski]

Yep :) Although it doesn't have to be the last bit of data. Netflix for example does this in their "Top 10" display, where they want you to explore at least one page before transitioning to unordered infinite.

My ideal way to implement this would be with a sentinel, but that may be challenging to place in JSX.

[nwidynski]

Hopefully answered by the other comments, but TLDR; A presentational carousel is basically just a useSelectableCollection which has given up control over its scroll offset to ScrollManager.

ARIA semantics of the controlled collection are largely left untouched, depending on how we decide to model the relationship between picker dot and controlled item/scroll target. As you already suspected, the essence of a carousel lies in the ARIA semantics of controls and their announcement of scroll/visibility state.