Do computed state properties cause over-observation?

[gerfarfel]

Prior to updating my app to TCA 1.7, I had thought that it was perfectly reasonable to use light-weight computed properties in State – but is it really?

Background

My app mostly uses SwiftUI, but there are a handful of UIKit views (typically using UIViewControllerRepresentable). Per the TCA 1.7 migration documentation, I switched from using Combine publishers to observe state changes to the new TCA observe method. I quickly noticed that the onChange handler was being invoked far more often than I expected it to be. The culprit seems to be the use of computed state. Even when no values change, the mere act of invoking a setter in a computed property triggers the observation handlers. The over-observation doesn't occur if no computed properties are used (going all the way back to the root Store).

Questions

1. Is it true that computed properties can cause over-observation when using the observe method?

2. If so, is there also a problem when using computed properties in SwiftUI views?

3. The documentation seems to imply that computed properties (even light-weight ones) are discouraged - is that a fair interpretation? In particular, the Performance/Store scoping section of the documentation states:

The most common form of scoping, that of scoping directly along boundaries of child features, is the most performant form of scoping and is the intended use of scoping. The library is slowly evolving to a state where that is the only kind of scoping one can do on a store. (emphasis mine)

4. A typical scenario for using computed properties is a view with several complex sub-views (tool panels, settings panels, and the like) that are all visible at the same time. Each sub-view is implemented as a separate SwiftUI View, with its own Reducer, scoped down from the parent Reducer. Because the sub-view requirements are overlapping, the only practical options are: 1) use computed properties on the parent state to generate the sub-view states, or 2) pass the entire (unscoped) parent state to the sub-views (obviously not ideal). What's the recommended way to handle this scenario?

I have a feeling that I'm probably misunderstanding something here. I'd appreciate some additional insight into these issues. Thanks!

[mbrandonw]

Hi @gerfarfel, we need more information to know what is going on in your situation. Ideally you would provide a minimal project that demonstrates some over-observing so that we can understand.

But, here are a few remarks based on your description so far:

Prior to updating my app to TCA 1.7, I had thought that it was perfectly reasonable to use light-weight computed properties in State – but is it really?

Lightweight computed properties are totally fine, even in 1.7.

Is it true that computed properties can cause over-observation when using the observe method?

No, the minimal amount of state is observed based on what state is accessed, regardless of computed properties or direct access to stored properties.

If so, is there also a problem when using computed properties in SwiftUI views?

No, computed properties are fine to access in SwiftUI views.

The documentation seems to imply that computed properties (even light-weight ones) are discouraged - is that a fair interpretation? In particular, the Performance/Store scoping section of the documentation states:

This documentation is specifically about scoping of stores with computed properties. For example, store.scope(state: \.someComputedField, action: …). And further, the docs are only about this kind of scoping when someComputedField is heavyweight, not light. Are you performing this kind of scoping?

A typical scenario for using computed properties is…

It's hard to say without a project to look at. Can you make a minimal project demonstrating the pattern you are currently following and then we can see what may be going wrong?

[gerfarfel]

Hopefully I haven't oversimplified this – it's not very representative of my real-world app, but I think it demonstrates what I'm asking about. In viewDidLoad, I'm only observing the color property. The state is generated by a computed property in the root reducer. Tapping the red SquareBoxView sends an action that toggles the isSelected property but does not change the color property; however, the observe block is executed (as evidenced by the print statement). Why?

import ComposableArchitecture
import SwiftUI

// MARK: - SquareBox

@Reducer
struct SquareBox: Reducer {
  @ObservableState
  struct State: Equatable, Identifiable {
    var id: UUID
    var color: UIColor
    var isSelected: Bool
  }

  enum Action {
    case tapGesture
  }

  var body: some ReducerOf<Self> {
    Reduce { state, action in
      switch action {
      case .tapGesture:
        state.isSelected.toggle() // does not change the color
        return .none
      }
    }
  }
}

final class SquareBoxViewController: UIViewController {
  let store: StoreOf<SquareBox>

  init(store: StoreOf<SquareBox>) {
    self.store = store
    super.init(nibName: nil, bundle: nil)
  }

  override func viewDidLoad() {
    super.viewDidLoad()

    // Tap gesture toggles the selection state but doesn't change the color.
    let tapGesture = UITapGestureRecognizer(
      target: self,
      action: #selector(Self.handleTapGesture(_:))
    )
    view.addGestureRecognizer(tapGesture)

    // Observe the color. This should only be seen once, right?
    observe {
      let color = self.store.color
      self.view.backgroundColor = color
      print("color changed:", color)
    }
  }
  
  required init?(coder: NSCoder) {
    fatalError("init(coder:) has not been implemented")
  }

  @objc func handleTapGesture(_ gestureRecognizer: UITapGestureRecognizer) {
    store.send(.tapGesture)
  }
}

struct SquareBoxView: UIViewControllerRepresentable {
  let store: StoreOf<SquareBox>

  func makeUIViewController(context: Context) -> SquareBoxViewController {
    return SquareBoxViewController(store: store)
  }

  public func updateUIViewController(
    _ uiViewController: SquareBoxViewController,
    context: Context
  ) {
    // nothing to do here
  }
}

// MARK: - Root

@Reducer
struct Root: Reducer {
  @ObservableState
  struct State: Equatable {
    let id = UUID()
    var color: UIColor = .red
    var isSelected: Bool = false

    // Computed property.
    var squareBox: SquareBox.State {
      get {
        return SquareBox.State(id: id, color: color, isSelected: isSelected)
      }
      set {
        if color != newValue.color {
          self.color = newValue.color
        }
        if isSelected != newValue.isSelected {
          self.isSelected = newValue.isSelected
        }
      }
    }
  }

  enum Action {
    case squareBox(SquareBox.Action)
  }

  var body: some ReducerOf<Self> {
    Scope(state: \.squareBox, action: \.squareBox) {
      SquareBox()
    }
  }
}

struct RootView: View {
  let store: StoreOf<Root>

  var body: some View {
    SquareBoxView(store: store.scope(state: \.squareBox, action: \.squareBox))
      .aspectRatio(1, contentMode: .fit)
  }
}

// MARK: - The App

@main
struct TheApp: App {
  var body: some Scene {
    WindowGroup {
      RootView(
        store: Store(initialState: Root.State()) {
          Root()
        }
      )
    }
  }
}

[mbrandonw]

Hi @gerfarfel, thank you for providing an easily runnable demo. That definitely makes it clear what is going on.

And yes, this kind of computed state is definitely something you are going to want to get away from in the observation world.

The problem is that SquareBox.State does not exist in the root state as a concrete thing, but rather is computed. This means the seemingly innocent act of accessing state.color is not so innocent. Under the hood you are actually going through Root.State.squareBox.getter, and hence touching all of id, color, isSelected inside observe { … }, and therefore observe thinks you want to be notified of all changes to those properties. That is why you are seeing the closure get called more often than you think it should.

The docs you mentioned previously about performance concerns around store scoping with computed property is from the 1.5.x release of the library, before observation tools. That advice is correct for TCA in versions 1.6.x and earlier. But in 1.7, scoping stores with computed properties is a lot more fraught due to the implicit observation mechanism. It really should be avoided entirely, and perhaps we can update the docs to reflect that.

If you can update your demo app to be a little more similar to your real app, perhaps showing 2 child features, then we might be able to give advice on how to avoid the computed properties.

[gerfarfel]

Thanks @mbrandonw, that helps clear things up. It sounds like the issues with computed state in the observation world pertain equally to SwiftUI views – is that correct?

Here's a highly simplified example of the type of data structures I have in my app. (Not a working demo – just a skeleton.) The parent state contains some large complex data structures. The two embedded child views don't need all of the data – they use overlapping subsets that is pulled from a variety of sources, including top-level parent state fields, computed fields, and from inside larger parent structs.

Passing the entire parent store to the embedded child views seems like a step in the wrong direction, but what's the alternative if computed state is off the table? I'll be very interested in your thoughts. Thanks very much!

@Reducer
struct Parent: Reducer {
  @ObservableState
  struct State: Equatable {
    var document: TileDocument          // a struct with a couple of dozen fields
    var tiles: IdentifiedArrayOf<Tile>  // an array of large, complex structs
    var selectedTileId: Tile.ID?
    var child1Params: Child1.Parameters // child1-specific parameters
    var child2Params: Child2.Parameters // child2-specific parameters

    // Computed properties

    var selectedTile: Tile? {
      guard let selectedTileId else { return nil }
      return tiles[id: selectedTileId]
    }

    var child1: Child1.State {
      get {
        return Child1.State(
          parameters: child1Params,
          selectedTile: selectedTile,
          selectedTool: document.selectedTool
        )
      }
      set {
        // ...
      }
    }

    var child2: Child2.State {
      get {
        return Child2.State(
          parameters: child2Params,
          selectedTile: selectedTile,
          viewCenter: document.viewCenter,
          viewScale: document.viewScale)
      }
      set {
        // ...
      }
    }
  }

  enum Action { /* ... */ }
  var body: some ReducerOf<Self> { /* ... */ }
}

@Reducer
struct Child1: Reducer {
  struct Parameters: Equatable {
    var isHidden: Bool
    var isLocked: Bool
  }

  @ObservableState
  struct State: Equatable {
    var parameters: Parameters
    var selectedTile: Tile?
    var selectedTool: Tool
  }

  enum Action { /* ... */ }
  var body: some ReducerOf<Self> { /* ... */ }
}

@Reducer
struct Child2: Reducer {
  struct Scratch: Equatable {
    // ...
  }

  struct Parameters: Equatable {
    var isScrolling: Bool
    var scratch: Scratch?
  }

  @ObservableState
  struct State: Equatable {
    var parameters: Parameters
    var selectedTile: Tile?
    var viewCenter: CGSize
    var viewScale: CGFloat
  }

  enum Action { /* ... */ }
  var body: some ReducerOf<Self> { /* ... */ }
)

[stephencelis]

@gerfarfel In addition to @mbrandonw's post, I think the most important question is if you're actually observing a performance issue, or if you just noticed that a closure was being evaluated more often than before. Is your application still performing the same as far as the end user is concerned, or has migrating to 1.7's observation APIs introduced a noticeable regression in performance?

As fussy as view stores and Combine publishers were, they did allow you to have absolute granularity in minimizing updates, even within a single view/controller, by deduping based on equality. But just because these updates could be scoped and minimized doesn't mean that the deduping didn't introduce its own performance issues.

Swift observation, on the other hand takes a much lighter weight approach. Given a scope that accesses properties, it will reevaluate that scope if any of the properties are touched in a mutating scope, whether or not they change. Folks even asked on the forums if observation should further de-dupe based on equality, and the team replied that the de-duping operation was often more heavyweight than minimizing the re-computations themselves.

So our changes align TCA observation more closely with vanilla observation, which we think is a good thing, and hopefully doesn't affect performance, but if it does, you have the same strategies you'd have in a vanilla observable app: you can take an observable scope (e.g. a SwiftUI view body, or an observe closure) and break it down into smaller components that access fewer properties and as such are recomputed less often.

[gerfarfel]

Let me throw out a more detailed (but still simplified) example. I'm hoping to get some ideas on how to avoid scoping on computed state. Start with underlying data structures for an animation app. At the root level is an array of Projects. Each project contains an Animation along with document structures describing the state of the project editor view. Animations contain an array of Frames. Each frame contains an array of Layers. Each layer contains an array of Shapes.

import Tagged

struct Shape: Equatable {
  // shape data: path, color, etc.
}

public enum LayerTag {}
public typealias LayerID = Tagged<LayerTag, UUID>

struct Layer: Equatable, Identifiable {
  var id: LayerID
  var shapes: [Shape]
}

struct Frame: Equatable, Identifiable {
  var id: Tagged<Self, UUID>
  var bounds: CGRect
  var layers: IdentifiedArrayOf<Layer>  // assume that the number of layers is fixed
  var thumbnailData: Data
}

struct Animation: Equatable, Identifiable {
  var id: Tagged<Self, UUID>
  var bounds: CGRect
  var frames: IdentifiedArrayOf<Frame>
}

enum DrawingTool { case brush, eraser, marker, pencil }

struct AnimationDocument: Equatable {
  var selectedDrawingTool: DrawingTool
  var selectedFrameId: Frame.ID
  var selectedLayerId: LayerID
  var viewCenter: CGPoint
  var viewScale: CGFloat
}

struct LayerDocument: Equatable, Identifiable {
  var id: LayerID
  var isHidden: Bool
  var isLocked: Bool
}

struct Project: Equatable, Identifiable {
  var id: Tagged<Self, UUID>
  var animation: Animation
  var animationDocument: AnimationDocument
  var layerDocuments: IdentifiedArrayOf<LayerDocument>  // assume one document for each frame layer
}

struct AppFeature: Equatable {
  var projects: IdentifiedArrayOf<Project>
}

In this example, the animation editor view is built up from several distinct UI elements, including the drawing canvas, a frame selector, a layer selector, a tool selector, settings panels, and more. That's a lot, I know, but if you're targeting a device with a larger screen (Macs, iPads), it's not untypical. The LayerSelectorView might be implemented as a SwiftUI view containing an VStack with an array of LayerPropertiesViews. The FrameSelectorView might be implemented with a UICollectionViewController, with a cell containing a FramePropertyView for each frame. Here are skeletons for a few of the Reducers that might be required.

@Reducer
struct Canvas: Reducer {
  @ObservableState
  struct State: Equatable {
    var frame: Frame?
    var selectedDrawingTool: DrawingTool
    var viewCenter: CGPoint
    var viewScale: CGFloat
  }

  enum Action {}
  var body: some ReducerOf<Self> { EmptyReducer() }
}

@Reducer
struct LayerSelector: Reducer {
  @ObservableState
  struct State: Equatable {
    var layerProperties: IdentifiedArrayOf<LayerProperties.State>
    var selectedLayerId: LayerID
  }

  enum Action {}
  var body: some ReducerOf<Self> { EmptyReducer() }
}

@Reducer
struct LayerProperties: Reducer {
  @ObservableState
  struct State: Equatable, Identifiable {
    var layer: Layer
    var layerDocument: LayerDocument?
    var isSelected: Bool

    var id: LayerID {
      return layer.id
    }
  }

  enum Action {}
  var body: some ReducerOf<Self> { EmptyReducer() }
}

@Reducer
struct FrameSelector: Reducer {
  @ObservableState
  struct State: Equatable {
    var frameProperties: IdentifiedArrayOf<FrameProperties.State>
    var selectedFrameId: Frame.ID
  }

  enum Action {}
  var body: some ReducerOf<Self> { EmptyReducer() }
}

@Reducer
struct FrameProperties: Reducer {
  @ObservableState
  struct State: Equatable, Identifiable {
    var frame: Frame
    var isSelected: Bool

    var id: Frame.ID {
      return frame.id
    }
  }

  enum Action {}
  var body: some ReducerOf<Self> { EmptyReducer() }
}

In my current style of implementation, I would use computed state to generate the various view states, like so:

@Reducer
struct ProjectEditor: Reducer {
  @ObservableState
  struct State: Equatable {
    var project: Project

    // computed state

    var selectedFrame: Frame? {
      return project.animation.frames[id: project.animationDocument.selectedFrameId]
    }

    var canvas: Canvas.State {
      get {
        let document = project.animationDocument
        return Canvas.State(
          frame: selectedFrame,
          selectedDrawingTool: document.selectedDrawingTool,
          viewCenter: document.viewCenter,
          viewScale: document.viewScale
        )
      }
      set {
        if let frame = newValue.frame {
          self.project.animation.frames[id: frame.id] = frame
        }
        var document = project.animationDocument
        document.selectedDrawingTool = newValue.selectedDrawingTool
        document.viewCenter = newValue.viewCenter
        document.viewScale = newValue.viewScale
        self.project.animationDocument = document
      }
    }

    var frameSelector: FrameSelector.State {
      get {
        let selectedFrameId = project.animationDocument.selectedFrameId
        let properties = project.animation.frames.map { frame in
          return FrameProperties.State(
            frame: frame,
            isSelected: frame.id == selectedFrameId
          )
        }
        return FrameSelector.State(
          frameProperties: IdentifiedArray(uniqueElements: properties),
          selectedFrameId: selectedFrameId
        )
      }
      set {
        // This is tricky: the frame selector can add, delete, and rearrange the frames.
        let frames = newValue.frameProperties.map { $0.frame }
        self.project.animation.frames = IdentifiedArray(uniqueElements: frames)
        self.project.animationDocument.selectedFrameId = newValue.selectedFrameId
      }
    }

    var layerSelector: LayerSelector.State {
      get {
        let selectedLayerId = project.animationDocument.selectedLayerId
        let selectedFrameLayers = selectedFrame?.layers ?? []
        let properties = zip(project.layerDocuments, selectedFrameLayers).map { document, layer in
          assert(document.id == layer.id) // assumed for this example
          return LayerProperties.State(
            layer: layer,
            layerDocument: document,
            isSelected: layer.id == selectedLayerId
          )
        }
        return LayerSelector.State(
          layerProperties: IdentifiedArray(uniqueElements: properties),
          selectedLayerId: selectedLayerId
        )
      }
      set {
        // assume that the layer selector only modifies layer properties, but not the layer shapes
        for properties in newValue.layerProperties {
          project.layerDocuments[id: properties.id] = properties.layerDocument
        }
        self.project.animationDocument.selectedLayerId = newValue.selectedLayerId
      }
    }
  }

  enum Action {}
  var body: some ReducerOf<Self> { EmptyReducer() }
}

Now that TCA is using observation, the use of computed state causes small changes to be propagated out to far too many observers. For example, if the layer properties view enables the user to mark an individual layer as locked, that change would propagate back from LayerProperties to LayerSelector to ProjectEditor, causing all of the layer property views to be refreshed. In the case of the frame selector view, modifying the current frame selection would cause all frame properties views to be refreshed. If the frame selector view is implemented as a UICollectionViewController, a simple selection change would lead to the entire layout being refreshed.

Under pre-1.7 versions of TCA, the combination of cached stores and the use of Combine for UIKit observation meant that the implementation using computed state worked fine. Perhaps there were some inefficiencies issues under the hood, but I didn't see them, and the performance of the app was excellent. With TCA 1.7, the over-observation caused by the computed state is causing huge performance problems. Currently, I don't see an alternative to pushing the entire project state down to all of the individual view components (along with additional view parameters, like frame and layer ID's, yuck). Maybe there's another approach I'm not seeing, perhaps involving a complete redesign of the underlying data structures.

I'll appreciate any direction @mbrandonw, @stephencelis, and the community can provide on this. Thanks!

[mbrandonw]

Hi @gerfarfel, I think there is no good solution to this right now. There is just an insurmountable impedance mismatch between value types and sharing state. Value types aren't meant for being shared. They get copied when you pass them around. And sure computed properties got the job done in a pre-observation world (though it was a bit of a mess), but now that no longer flies with observation tools.

Luckily we do have a solution coming. We will be releasing details for how to handle shared state in TCA soon, and it's only possible thanks to observation tools in Swift. So I would say you may need to hold off on the observation tools for the time being and then take another look once we release the new state sharing tools.

[gerfarfel]

Okay, thanks again for the help @mbrandonw (and @stephencelis) – I really appreciate it. I'll follow your advice and back off on the observation tools for now. And I'll look forward to hearing about the solution! (I also understand what it's like to build tools and then have people use them in ways that don't quite match up with the original intent.) Despite the occasional hiccup, working with TCA is a real pleasure, and the PointFree video series (along with TCA) has probably improved my coding skills more than any other single thing I've done in my career.

One other thought, for what it's worth... This is a gross generalization, but it seems to me that well-designed iPhone apps are likely to segregate functions into separate views (because of screen size), whereas Mac and iPad apps may incorporate more functions into a single view (e.g., settings panels and complex UI elements). Virtually all of the examples used in the PointFree video series are shown on an iPhone. That's understandable, but I wonder if that focus helped to obscure (from me, at least) the issues we've been discussing here.

[mbrandonw]

While Mac and iPad apps may exacerbate the problem of shared state more, we are certainly aware of the problems and iPhone apps definitely run into this issue a lot too. Even the SyncUps demo app in the repo could greatly benefit from shared state. Currently it has to employ tricks to synchronize value types across multiple features, and ideally none of that would be necessary.