Testing load/subscribe errors

[seanmrich]

I'd like to exercise errors in persisted shared values in previews and tests. I see that you can set a custom value by setting the default:

@Shared(.fileStorage(url))
var persisted: Value = .init()

But if I want to test what happens when the persistence strategy throws an error in the load or subscribe methods, how do I do that?

[mbrandonw]

Hi @seanmrich, I don't think that is currently possible. The FileStorage type is what abstracts over the file system right now, but there is no public initializer and only two public values: .fileSystem and .inMemory. We would need to open this up more to allow providing a custom file storage system that throws errors on save/load.

We'll have to think about it more.

[seanmrich]

Thanks, @mbrandonw. I was thinking it would be great to have some symmetry with the dependency system. I'm using a custom persistence strategy using GRDB based on your example code. I created a separate dependency that executes write operations on the database dependency. The extra layer of abstraction lets me override writes to the database in tests and previews, mostly to throw errors. I'd love to be able to do something similar when fetching.

Perusing the code, it looks like there's a central cache of the shared references. If there was a withShared cousin to withDependencies maybe it would allow setting these up by referencing the shared key?

Thanks again. Your tools have saved me so much time and frustration -- can't thank you guys enough.

[mbrandonw]

I'm not sure opening up the references to the outside would be good overall, and not sure how it helps in this situation. At the end of the day, you are wanting to change the behavior of .fileStorage, and that I think that can only be done by tweaking the FileStorage dependency under the hood.

I also don't think we want to open up that dependency's interface fully because it's quite ugly, and we aren't confident it has the correct shape right now. But, I could see us at least making the inMemory strategy more flexibly by providing optional errors to use when loading or saving. Something like this:

#Preview {
  _ = prepareDependencies {
    $0.defaultFileStorage = .inMemory(loadError: SomeError())
  }
}

Would that help in your situation?

[seanmrich]

This might not make sense for the package and may not even be possible, but I was thinking of something more targeted. Imagine I have a feature whose state tracks two persisted values, each using a different FileStorageKey. I'd like to be able to preview and test each in a failure state. There are cases where the UI and behavior are different in a single feature so specificity would be very useful.

Perhaps the SharedReaderKey could have a failure condition that could be assigned along with the code that sets a value for previews/tests:

@Shared(.fileStorage(urlA)) var count: Int = 5
$count.error = SomeError() // Probably stored as any Error

Any attempts to load or yield a subscribed value would be overridden by throwing the error. This is probably something I can do with my own custom persistence strategies, but obviously not the built-in ones.

[mbrandonw]

The inMemory(saveError:loadError:) example I sketched out could also take closures for each argument that is handed the URL that is being saved/loaded. That would allow inspecting the URL to determine when/if to throw an error.

In general, I think it is always going to be up to the persistence strategies to provide the tools for robust testing. Already they are responsible for designing dependencies to use under the hood (such as FileStorage and defaultDatabase), and so they will just need to go the extra mile to make it possible to override those dependencies with specific error states.

I do not think we want to allow anyone to reach into any shared state and put it into an error state.

[seanmrich]

Makes sense. I’ll give it a try with my own and see how it goes. Thanks again!