Model Selection Queries

[bosconian-dynamics]

I'm not too familiar with AI Horde yet and I've only recently begun reading up on the TOP 10 debate, and the need for NEW N etc., but it all makes me wonder if support for a more robust model query logic might be favorable. It seems like the list of desirable "meta instruction" selectors could continue to expand, and it could be convenient to facilitate combinations of criteria (e.g. TOP 10 NEW SMALL NONXL SFW, as an extreme).

This could take the form of an esoteric query language - perhaps even derived from the current meta instructions. But I can't think of a reason to not support a structured query object directly, given that should a query language come to be it would likely make sense to parse it into such an object to pass as an input to consolidated selection/filtration logic, anyway.

In my head, it looks something like this (albeit some fairly convoluted examples):

# "<=25 most popular non-xl, non-SD2 SFW models this month, smaller than 4GB a piece"
- query:
    ref.size_on_disk_in_bytes:
      lt: 4000000000
    ref.baseline:
      not:
        - "stable_diffusion_xl"
        - "stable diffusion 2"
    ref.nsfw: false
  limit: 25
  orderby:
    stats.generations.month: "DESC"

# "<=100GB total of the smallest models in the reference under 3gb a piece"
- query:
    ref.size_on_disk_in_bytes:
      lt: 3000000000
  limit:
    ref.size_on_disk_in_bytes: 100000000000
  orderby:
    ref.size_on_disk_in_bytes: "ASC"

# "All the ponies"
- query:
    ref.name:
      pattern: "\b[Pp]ony"

# "The 11th-25th least popular models in use which are either XL, NSFW, anime-styled, or any combination of the three"
- query:
    or:
      ref.baseline: "stable_diffusion_xl"
      ref.nsfw: true
      ref.style: "anime"
    stats.generations.month:
      gt: 0
  limit: 15
  offset: 10
  orderby:
    stats.generations.month: "ASC"

This was inspired by #382 and #353, and would be one means of pretty directly facilitating the objective stated in #267. It doesn't diminish the need for or utility of meta instructions, but may simplify implementation of more complex instructions moving forward in exchange for some added complexity for the near future. I think it may have some limited potential in enabling workers to provide more adaptive model selections in response to changing conditions, as well.

[tazlin]

I like the idea here. The system as described above, power users would likely benefit in the short term, and further out this sort of querying system would be best abstracted through some sort of GUI. There is already an active effort to create a launcher (with some degree of configuration editing) and there are medium-long term plans for a native GUI as well.

A first step might be to replicate all of the existing meta commands and model them using the selected query language, at which point the existing config loader could translate the meta commands into looking up the query instead. Then, it could be extended to other presently unsupported ones as noted in the opening issue comment.

For reference:

• The meta strings are defined here in bridge_data.py in the sdk
• The class that resolves model meta commands to models is here in model_meta.py in the sdk
  • This class could be forklift replaced with the proposed query-supporting system as long as the function signatures are maintained.

[bosconian-dynamics]

Awesome!

This is something I will poke around with a bit, but I'd still encourage any other interested party to work on their own implementation. I'm not familiar with any of the relevant techs and my spare time is constrained for the next couple months, so my exploration here will likely be very slow, and my own attempt likely requiring a fair bit of iteration.

Offloading this query logic to some UI down the road definitely makes a lot of sense to me - and I'll definitely keep an eye out for those UI efforts in the GitHub repos, moving forward.