What would it take to move from random commits to versioned releases with at proper test coverage?

[Tikonderoga]

First of all - thank you all contributors. It's an amazing project!

But, I can't help but notice how often daily commits break this and that functionality. I have a version from the beginning of January and simply hesitant to update it, because every day I see new bug reports about certain features broken.

Perhaps I didn't look well enough, but I can't find any information about any test coverage for the project. I think that adding some (at least few manual scenarios, verifying some predefined set of features) and running it before the release would reduce the amount of frustration for users and number of bug reports like "it was working yesterday, but not anymore".

I'm willing to help with test coverage. I could outline test scenarios and can run them, or, at least portion of them, on regular basis. However, I'm not really familiar with the process of adding any QA to open source projects like that. So, any ideas and feedback will be appreciated.

[Gerschel]

Branching the repo into versions at regular release points, maybe monthly at first (weekly? depends on burnout rate).
This can be automated.
Remove code that autoupdates some things.
No major commits and no major features a few days before this commit, that way it gives a chance to find and fix some bugs.
Bug fixes on main branch should be pushed to feature branch, or vice versa, prioritize bug fixes over features, don't mix them in an update.
Assign areas of the code/feature (a lot of features are spread thin over different files) to people that wont to take on the responsibility, but still allow other contributors to that code section.
Other pending features can be held in a feature branch until this commit, then they can be merged into master.
Communicate this date with extensions writers, use a version number that they can include in a branch, so core releases version branch can pull from the correct extension branch that supports it.
I think this would keep extension writers from burnout, they have a month (or week or whatever) before having to worry about keeping the extension updated, instead of nightly. Gives them some leisure time to get their extension bug free and to incorporate their features (it sucks to not be able to add your features into your extension because you're busy keeping it functional, becomes a chore).
If we had scheduled releases with a no new features period, then this period can be used to write some tests that can check extension compatibility.

You can try and fork the repo and tackle this with some volunteers.

The challenge would be, most want the newest thing, and they want to do different workflows.
You'd have to get other extensions on board and have them polish to a certain webui version, or do all the work yourself. @ClashSAN started a repo for extensions to be able to say which commits of their extension works with which commits of the webui. So it could be a good starting point for this information. But you'll see that most have not contributed the state of their extension, myself included, personally, I dropped support of my extensions at this time because of burnout ( I would really like to target version releases). https://github.com/ClashSAN/sd-webui-extension-records

If you have an extension you are using, and it's not part of the extension records, and you know it's working bug free, submitting it to the extension records would help. If you have time, you can test the ones posted there and update accordingly.

[Tikonderoga]

That looks like a solid plan / template, thank you @Gerschel

@AUTOMATIC1111 - would you be interested in moving to that direction at all?