Alternative to observe: track what values get accessed from ViewStore inside WithViewStore

[fabstu]

Here I already mentioned my foolish idea to try to track what values get accessed inside a WithViewStore closure run.

Its pretty easy to detect what gets accessed through the subscript on the ViewStore, as long as .state is not used directly.

Correction: its not as simple to track what is being accessed as I thought.

Proof of concept

Anyway, here is a small proof-of-concept. It avoids re-running the closure as long as only equatables changed, but re-runs when any non-equatable changed (didn't test the second part).

• Branch (based on protocol-beta): https://github.com/fabstu/swift-composable-architecture/tree/exp-tracking-changes
  Added the example as the first case study "Smart View Store".

• Commits: protocol-beta...fabstu:swift-composable-architecture:exp-tracking-changes

Tracking:

Hacked in

• subscript using KeyPath
• .binding
• .state
• subscript with HashableWrapper

Still missing:

• yield / send/(_:while:)
• publisher

Limitations

• only tracks equatable access on the first layer
  • tracks changes to more access than necessary, for example accessing .tabs.count tracks changes to any tab in .tabs instead of just a changed count (Int).
  • first layer could be unequatable, while layer could have been equatable -> unnecessary re-runs
• when accessed first-layer value is unequatable, have to re-run.
• direct access via .state and similar ways without going through dynamic member lookup means any variable of state could have been accessed, requiring a re-run without any tracking gains while the overhead from the dynamic member lookup still exists (can be optimized?).
• overhead of storing access method (AnyKeyPath, scoping function) and variable value on each variable access from comparisons otherwise unnecessary when properly scoped
• overhead when the state changed from comparisons between the stored and the current tracked values.
• does not yet reset the comparison state before the closure re-runs.
• Access not tracked yet:
  • yield / send/(_:while:)
  • publisher
  • ... other unknown?
• what is tracked is just hacked in

Note: This is just an inspiration. Did you guys try something like this already? If so, I'm just wondering what came of it.

Open questions

• when a variable of the state is only accessed through a binding: do we re-render the closure when the ViewStore state changes, or can we notify SwiftUI another way that only the locations with @Binding or some such places should be re-run?
• does every place state is used need the access recording?
• can we centralize the recording by wrapping at every place that accesses the state directly to use a property wrapper recording access instead?
• does SwiftUI have some magic way to otherwise get more localized updates => can we limit objectWillChange.send() calls in other places

[mbrandonw]

Hi @fabstu, thanks for experimenting with this! Could you push a branch that has these changes so that it's easy for others to poke around?

[fabstu]

Hi brandon, I added the branch in the description above.

note: It is just supposed to be an inspiration. I do not know about any repercussions that come from such a change. I just wanted to see whether it can work at all :)

[mbrandonw]

Great, thanks for sharing! A few things come up while thinking about this:

• If you need to access a sub-property of a property, then you will accidentally observe the full property. Taking the tab-based example from the performance article, if you wanted to show a badge on the activity tab using state from the activity like this:

  WithViewStore(self.store) { viewStore in
    TabView(selection: viewStore.binding(send: AppAction.tabSelected)) {
      ActivityView(
        store: self.store.scope(state: \.activity, action: AppAction.activity)
      )
      .tag(AppState.Tab.activity)
      .badge(viewStore.activity.activityCount)
      …
    }
  }

  …then you are now observing all of activity even though you just want the count.

• Because view stores can be created repeatedly for the same view, the comparisonState is going to be cleared out often. I believe that means it's possible to get into a state where you are no longer observing state for something that is visible on the screen.

  In fact, the todos and voice memos demos don't work with this change, and it may be related to that, but I'm not entirely sure.

It's a very promising exploration, but some more research needs to be done. Thanks again for taking the time to share.

[fabstu]

…then you are now observing all of activity even though you just want the count.

Yes. Sadly, haven't found any way to give back a value that records back what was accessed. This might only work by giving back a Wrapper, maybe similar to a binding, where people access the final value using .wrappedValue or similar. Swift one day supporting the embedding of access tracking while giving back the same exact type that was requested would be cool, but I'm not holding my breath.

In fact, the todos and voice memos demos don't work with this change, and it may be related to that, but I'm not entirely sure.

Its related to the fact that tracking is not as easy as I thought.

Were missing:

• .binding
• .state
• subscript with HashableWrapper

Are still missing:

• yield / send/(_:while:)
• publisher

With binding, state and the other subscript being tracked now, Todos seems to work. However, I'm not even sure they are supposed to be tracked the way there are now, since I just hacked it in the way I thought it might work.

[fabstu]

If you need to access a sub-property of a property, then you will accidentally observe the full property.

This might be fixable when we make the user adapt binding chaining like with $state.value1.value2 like viewStore.value1.value2.state instead of viewStore.value1.value2, where the resulting value is a Binding/Wrapper instead of the real value. Since we can pass the prior KeyPath and the ComparisonState into each piece of the chain we might be able to chain the KeyPath together? The idea is to give back a wrapped value in each dynamic member lookup, until at the end where the user retrieves the value with .wrappedValue, the access to which stores its returned value and its keypath inside ComparisonState.

That's open for research. The UX would be different, but could be doable. We can rename the last step to .state as well, since we are not bound to use a property wrapper and can so return the wrapped value.

edit: That might actually be how bindings (stored in the property wrapper @Binding) that are the only way a @State is accessed can be invalidated alone, without re-running the view body containing the @State itself (assuming @State only gets accessed through the binding and not directly using .wrappedValue).

It could be an experiment to try to actually use the Binding<> type and store its .objectWillChange alongside the scope-method/AnyKeyPath and alongside the value itself to trigger that instead of a whole closure re-run. Well, that maybe won't work and would need a custom Binding-type and property wrapper, since there is no method to propagate the ComparisonState and the KeyPath, even if something similar is embedded internally by the binding returned by @State.

[tgrapperon]

Hey @fabstu! Sorry, I misread your intent when you proposed the idea in the closed PR. This is indeed related to something with which I've experimented in the past. In one case, I was building the list of accessed properties as you did (but I was only keeping track of the KeyPaths, not the accessed values, and I don't think I pushed it publicly), and in the other, I used them to diff with non-accessed properties to suggest using a ViewState in #1225.

My main concern is performance, as storing and rehydrating a collection of KeyPath to compare is likely more expansive than comparing two Equatable value-types once. Your approach is more subtle than mine, as you keep one value around (I think this is what they do in @Environment too in SwiftUI). It would be interesting to measure the difference with a proper ViewState. It is probable that this will work in many scenarios.

[tgrapperon]

If you realize benchmarks, you can probably remove the .removeDuplicates in ViewStore, as you're effectively performing the same work dynamically, or you can maybe bake your dynamic comparison as a removeDuplicate(by:) filter on Publisher.

[fabstu]

Hey @tgrapperon!

All good, all good. I have concerns about performance as well.

If you realize benchmarks, you can probably remove the .removeDuplicates in ViewStore, as you're effectively performing the same work dynamically, or you can maybe bake your dynamic comparison as a removeDuplicate(by:) filter on Publisher.

We can't filter out the state changes, since the new currently-unused state might become used when something outside of the viewStore changes => can't filter out before the sink to skip objectWillChange.send().

About duplicate work: We might betray the intention of the user to only trigger a re-drawing, when a certain variable changes here. This might make the use of removeDuplicates and the automatic detection together here surprising, or at least a change in semantics.

[fabstu]

and in the other, I used them to diff with non-accessed properties to suggest using a ViewState in #1225.

That sounds cool!

[GreatApe]

Haha, I went and wrote a very similar thing last night after our discussion. Also tried some concepts where you manually specify what you want to observe:

let titleStore = store.observe(\.title)

EDIT: I've moved the the rest, a proposal for an alternative solution, to a separate thread (#1454), so as to not hijack this one.

[GreatApe]

I think this is very cool, and as I said I explored similar things myself, but I just want to clarify, this would only be for debug builds, right? At least I personally don't think we should have all this magic in release builds, I think we should trust people to know what they're doing if they get that far.

But for debug builds, I do feel that this is a better way to nudge people than making observe: mandatory.

[fabstu]

Is re-running the body even when nothing that changed is used not surprising and magic to users in itself? Only enabling this for debug builds just makes users get hurt later.

Incase the approach is viable: An alternative may be to make observe: mandatory for WithViewStore and create a new WithSmartViewStore, where observe: is not required. For WithSmartViewStore the user knows that the view store tries to only update the body when it is really necessary.

We can also limit the allowed interactions for SmartViewStore by a bit to tell the user that certain interactions makes usage tracking impossible, you may as well use a WithViewStore. Guiding the user to use WithViewStore instead of WithSmartViewStorewhen tracking is useless due to the use of .state might be viable with runtime warnings as well.

[GreatApe]

Is re-running the body even when nothing that changed is used not surprising

For me, I would not expect the store to know what I'm accessing. If you have a vanilla SwiftUI view with a single viewState input parameter/property, it will not be smart about redrawing, as far as I know. So I would not expect the Store to be smarter than that. I would not want it to be, either. But using it to display runtime warnings in debug builds would be very helpful.