Implement a new TAS state manager

[SuuperW]

This PR writes a new state manager. It takes some inspiration from the current Zwinder manager, but has a few differences. The primary goal of this state manager was to enable capturing states out of order. This provides two primary benefits:

1. Users who load a branch, emulate forward a little, then rewind can still use the full buffer space.
2. It becomes much easier to "thin out" states (keeping only a subset of them) for saving and to then use them again on load. With the default settings, auto-save is now much less annoying.

In my very limited testing, performance seems virtually identical, except when re-playing an old part of a movie where it will be capturing states (what Zwinder calls "gap states"), for movies with a large number of states. That's slower due to a stupid of .NET's SortedSet which I haven't figured out a nice workaround for.

There is no support for loading states from or automatically migrating settings from the old manager. That doesn't seem super important, and anyway BizHawk already doesn't claim any backward compatibility in this area.

There is also no support for compressing states with ZStandard. This can be added, but is in my opinion very much not worth the performance loss.
There is also no support for TempFile states. I do plan on implementing this, but I currently don't see that as a high priority.

Check if completed:

• I have run any relevant test suites
• I, the commit author, have read the licensing terms for contributors (last updated 2024-06-22) and am compliant

[Morilli]

4KiB is 4096 bytes, is this a typo or intentionally -4?

[SuuperW]

Intentional. 4 bytes are for page metadata.

[nattthebear]

The zwinder was designed for the requirements and limitations of linear rewind, and has worked out well there. But it always seemed to be a bit of a mismatch for tasstudio, and I never quite understood the tasstudio requirements. So I'm not surprised you're able to come up with something better, and it will be good to replace this.

[SuuperW]

I don't think anybody quite understands the TAStudio requirements, because different users have different workflows, the same user may have different workflows for different games or TASes, and different consoles have very different limitations arising from state size and the time it takes to make states.

Although I believe this implementation will fit my use cases very well (and I've been using it a little since creating it), I would not be surprised if someone else wants some different behavior or options. (or even if I do if/when I TAS a new game/console) We'd need feedback from actual users to know that.

[vadosnaprimer]

One of my biggest problems with the current state manager (that it introduced compared to the decay algo I added years ago) is that if you go back and reemulate past states without changing any input, the very end of the greenzone (the future that has already been emulated) will not release states. That means if I watch some movie I'm judging and I notice a problem at frame 10k, and it's a system with big states, trying to go back to, say, frame 5k, I will only have A FEW states at my disposal. I may emulate frame 6k, then 6100, and trying to go back to frame 6k will launch the long seeking process from somewhere like frame 1k. Because the entire buffer is already fully occupied with all the states that are currently in the future compared to my playback position. To regain those states I have to change input, so those future states are erased and I can finally navigate comfortably.

The decay I added was not only thining out states from the far past but also from the far future. If there's little to no probability that I'll suddenly jump back to the future, I don't need all of those future states at all times. And when I need them, I'll reemulate them from some remaining keypoint states that are, say, 500 frames apart.

tl;dr: the area we're currently in needs the highest amount of states while we're navigating it, and farther away from it we should be gradually dropping more and more states.

I haven't checked the code for this PR, but how does it handle this problem?

[SuuperW]

First, your use case is not one I had considered. When I rewind to watch old sections it is while I am working on a TAS, and I absolutely will want to jump back to the present and keep states there. It's usually just to review how I did a certain trick at some earlier point or what my strategy was.

Now for how this state manager handles rewind and states in "gaps". The manager keeps three categories of states which are similar to the old manager: old (ancient), mid (recent), and new (current). It does not have a separate "gaps" category. When you rewind to watch an old section, it pulls states out of the "mid" category. If there are any mid states past the currently emulated frame it will take the oldest one of those. If not, it will take the oldest mid state behind current emulation. This enables you to rewind and keep a section of states that spans as many frames as what the "mid" category can handle.

This should in general be much more than the old manager's "gap" category but I am not sure it would suit your use case.
Is it be okay for your use case that it uses the interval for mid states (now called "Frames Between Mid States", user-set rather than computed based on state size and buffer size) rather than the interval for new states? You said the currently emulated region should have the highest density. I think this is at odds with what I want from rewinding (at least, removing 90% of "new" states while rewinding is), but maybe we could have more frequent states while watching old sections and thin out the "new" states to whatever the "mid" interval is? That would require significant changes to how it currently tracks states in order to do it efficiently.
Or, you might be able to make the current one work for you by setting the "new to mid ratio" to something very low. This will place most of the states in the "mid" category, leaving you with just a few "new" states at the end of the greenzone if you re-emulate a bunch of past states. But there still won't be any "thinning out" of those future states.

[vadosnaprimer]

I'll need some time on testing this with several different cores so see how it feels.

You said the currently emulated region should have the highest density. I think this is at odds with what I want from rewinding (at least, removing 90% of "new" states while rewinding is), but maybe we could have more frequent states while watching old sections and thin out the "new" states to whatever the "mid" interval is? That would require significant changes to how it currently tracks states in order to do it efficiently.

The most recent example of this wasn't even to just watch a movie in tastudio and try improvements that come to mind. With huge states, every time you have to go back to check or redo something that affects the future, you can find yourself in that situation I'm describing. Huge states saturate the buffer very quickly, so how distant that past you need to check is depends on state size and the amount of gigabytes you can afford to give them. If future states persist, precise navigation to find the right point when something in memory changes becomes much harder. You've just emulated a few frames so it's natural to think that it means they're still in the buffer, but in fact only one of them is, and to navigate in there you have to either rewind from a secondary class state every time, or change input every time to future states don't pile up.

With tiny states it's not a problem because you can afford having hundreds or thousands of them at all times.

Or, you might be able to make the current one work for you by setting the "new to mid ratio" to something very low. This will place most of the states in the "mid" category, leaving you with just a few "new" states at the end of the greenzone if you re-emulate a bunch of past states. But there still won't be any "thinning out" of those future states.

Removing first class states form a far future region and only leaving secondary class states there for backup would probably be ideal. Current TSM seems to only work in one direction, so it feels super weird to be unable to navigate around the place you've just been to and emulated it all. Maybe my whole vision of TSM is built around how states are captured in real time as opposed to emulated time. States that were captured an hour ago should be the first ones to go if we're out of capacity, regardless of their position. Then states captured 50 minutes ago, etc. If I need to really investigate some area, I only need to emulate it all from some keypoint (secondary or tertiary class states) and it's again all mine for a while, until I move to work on another section.

State decay was one solution to this but now thinking about it, I can say that it wasn't an exact implementation of this vision (because I never directly formulated it) but an incredibly close approximation.

[SuuperW]

I'll need some time on testing this with several different cores so see how it feels.

You posted this then edited your post. Is anything you say in that edit based on testing?

You've just emulated a few frames so it's natural to think that it means they're still in the buffer, but in fact only one of them is

Are there any existing cores with such huge states, that don't have AvoidRewind set? NDS is relatively large but still you get a decent number of states with the default buffer size. I am getting nearly 200 frames with "new" states (so nearly 200/4 = 50 states) with the default settings. Granted 200 is still not a lot, but probably enough to go back after accidentally overshooting the intended frame. Edit: For rewinding it'd use "mid" states. With default settings, that'd be about 24 states = 24 * 20 = 480 frames.

first class states ... secondary or tertiary class states

You are suggesting a new way of categorizing states, then? Based on "how states are captured in real time"?

State decay was one solution to this but now thinking about it, I can say that it wasn't an exact implementation

What do you mean by "state decay"? There is a concept of "state decay" in the Zwinder state manager, but it was highly similar to what this one does, not based on states' real creation time like you are talking about.

[SuuperW]

What do you think about always trying to keep enough states to satisfy "ancient interval" (from Zwinder) or "Frames Between Old States" (this manager)? Would you want to keep states such that you can always load one that is within X frames of where you want to go?

Perhaps the best solution to support watching movies like you want would be to implement another state manager that's based on state real creation time, and let the user choose which one they want?

States that were captured an hour ago should be the first ones to go if we're out of capacity, regardless of their position. Then states captured 50 minutes ago, etc.

Would you expect the oldest (by real time) state to be the first to go even if it's the only state within 1,000 frames behind the currently emulated frame?
If what you want truly is as simple as always removing the oldest state, by real time, that would be easy to implement as a new state manager.

[vadosnaprimer]

State decay is #1128

Will reply to the rest tomorrow.

Is anything you say in that edit based on testing?

No.

[vadosnaprimer]

Getting around to test this is taking a long time...

[SuuperW]

New commit makes it possible to have different movies with different state manager implementations.
There is no UI, and old movie files do not keep the old state manager without modification. You need to add "$type": "BizHawk.Client.Common.ZwinderStateManagerSettings, BizHawk.Client.Common" in the GreenZoneSettings.txt file.

[YoshiRulz]

	/// <summary>
	/// This class will manage savestates for TAStudio, with a very similar API as <see cref="ZwinderStateManager"/>.
	/// <br/>This manager intends to address a shortcoming of Zwinder, which is that it could not accept states out of order.
	/// Allowing states to be added out of order has two primary benefits:
	/// <list type="number">
	/// <item><description>Users who load a branch, emulate forward a little, then rewind can still use the full buffer space.</description></item>
	/// <item><description>It becomes much easier to "thin out" states (keeping only a subset of them) for saving and to then use them again on load.</description></item>
	/// </list>
	/// Out of order states also means we do not need separately allocated buffers for "current", "recent", etc.
	/// This gives us some more flexibility, but still there won't be any one-size-fits-all strategy for state management.
	/// <br/>For the initial implementation I will be using similar settings as Zwinder has, but this may change in the future.
	/// <br/>Additionally, this approach allows us to take the main goals of <see cref="ZwinderBuffer"/> into a state manager:
	/// <list type="number">
	/// <item><description>No copies, ever. States are deposited directly to, and read directly from, one giant buffer (assuming no TempFile storage, which is not currently implemented).</description></item>
	/// <item><description>Support for arbitrary and changeable state sizes.</description></item>
	/// </list>
	/// </summary>