Proposed RFC Feature Quality/Scalability Framework

[AMZN-alexpete]

Summary:

A quality/scalability framework should be created that gives developers the ability to apply settings based on the platform and device their application is running on.

What is the relevance of this feature?

Developers do not have a framework for specifying and automatically selecting settings presets based on hardware, capabilities and user preferences. The legacy system which used CVarGroups has been removed and is no longer supported.

Developers typically use this kind of feature to specify Low/Medium/High/VeryHigh quality settings for target platforms so a game or simulation will perform as intended with the expected quality.

Feature design description:

A quality/scalability framework that satisfies the requirements can be built using CVARs and the Settings Registry.

1. Settings are defined with CVARs, e.g. a rendering setting for turning on shadows named r_shadows
2. Groups of settings are defined in the Settings Registry e.g. a group named q_graphics could control all the graphics quality settings.
3. Rules for which settings to use for specific devices are defined in the Settings Registry
4. Settings and rules for specific platforms are defined in the Settings Registry using the Platform Abstraction Layer (PAL) folder structure.

Developers will be able to use the initial framework without any graphical tools or Editor menus, but those tools will be part of a future RFC.

Technical design description:

The technical design is mainly comprised of settings groups and levels, device attributes and device settings rules.

Settings groups and levels

Settings groups cvars can be defined (e.g. q_general, q_graphics, q_physics or GraphicsQuality, GeneralQuality etc.) and within each group, quality levels are defined (e.g. Low, Medium, High).

NOTE: the r_ prefix denotes rendering/graphics CVARs and the q_ prefix denotes CVARs or CVAR groups that control quality. These largely follow a pre-existing set of prefixes listed here: https://github.com/o3de/o3de/blob/development/Code/Legacy/CrySystem/ConsoleHelpGen.cpp#L672

Settings groups and quality levels are defined in .setreg files at the key /O3DE/Quality/Groups/\<group\>/Levels

File: O3DE/Registry/quality.setreg

{
  "O3DE":{
    "Quality":{
       "DefaultGroup":"q_general",  // default/fallback quality group
       "Groups":{
         "q_general": {
           "Description":"General quality group.  0 : Low, 1 : Medium, 2 : High",
           "Levels":[ 
            "Low",         // level 0 (based on array index)
            "Medium",  // level 1
            "High"        // level 2
          ],
          "Default": "High", // default level, can also be index if preferred
          "Settings": {} // Settings could go in here, but it's more likely those will be in Gems
        }
      }
    }
  }
}

Gems like Atom would define new settings groups and levels as needed.

File: O3DE/Gems/Atom/Registry/quality.setreg

{
  "O3DE": {
    "Quality": {
      "Groups":{
        "q_general":{
          "Settings":{
            "q_graphics":[0,1,2,2]  // q_graphics has one more level than q_general
          }
        },
        "q_graphics": { // General Graphics settings that uses shadow and visibility quality group settings
          "Description":"Graphics quality group. 0 : Low, 1 : Medium, 2 : High, 3 : VeryHigh",
          "Levels": [ "Low", "Medium", "High", "VeryHigh" ],
          "Default": "High",
          "Settings": {  // Settings could be defined in a separate file if desired like quality.graphics.setreg (see Settings example below)
            "q_shadows": [0, 1, 2, 3],    // q_shadows group cvar defined below
            "q_visibility": [0, 1, 2, 3]  // q_visibility group cvar defined below
          }
        },
        "q_shadows": { // Shadows Settings levels
          "Levels": [ "Low", "Medium", "High", "VeryHigh" ],
          "Settings": {
            "r_shadowResolution":[256, 1024, 2048, 4096] // actual shadow cvars
          }
        },
        "q_visibility": { // LOD/Visibility settings
          "Levels": [ "Near", "Medium", "Far", "VeryFar"], // different level names
          "Settings":{
            "r_viewdistance":[100, 256, 512, 2048] // actual visibility cvars
          }
        }
      }
    }
  }
}

Settings

Each setting is a CVAR (e.g. r_shadows, p_gravity etc.) and can be put in a group which denotes a cvar (e.g. q_graphics, q_physics, q_general) within a .setreg file at the address /O3DE/Quality/Groups/<group>/Settings
CVARs can be configured with flags based on the game needs to restrict who can change their settings and when.

File: O3DE/Gems/Atom/Registry/quality.setreg

{
  "O3DE":{
    "Quality":{
      "Groups":{
        "q_graphics":{
          "Settings":{
            "q_shadows":[1,2,3,4], // (compact form) graphics has 4 levels, so 4 values are defined, this compact form makes it easy to compare values for each level
            "r_sun":1, // (compact form) r_sun is 1 for all levels
            "r_example":{ "0":1, "Medium":2} // alternate form can target a level by index or name
          }
        }
      }
    }
  }
}

These registry files would exist in the Gems that provide the cvars, but can be overridden in the active project.

Settings can be overridden for specific platforms (e.g. iOS, Android, Linux etc) by placing the overrides in .setreg files in the appropriate PAL folder.

File: O3DE/Gems/Atom/Registry/Platform/Android/quality.setreg

{
  "O3DE":{
    "Quality":{
      "Groups":{
        "q_graphics":{
          "Default":"Low",
          "Settings": {
            "q_shadows":[0,1,1,1],  // lower shadow quality levels
          }
        },
        "q_shadows":{
           "Settings": {
            "r_shadows":[0,1,1,1] // no shadows at all on lowest
          }
        }
      }
    }
  }
}

Device attribute API

A DeviceAttributeRegistrar exists in AzFramework for registering device attributes and device attribute interfaces will be registered for model and RAM.
Device attribute names must be unique and are case-insensitive.

struct IDeviceAttributeInterface
{
    // get the name of the device attribute e.g. gpuMemory, gpuVendor, customAttribute42
    virtual AZStd::string_view GetDeviceAttribute() const = 0;
     
    // get a description about this device attribute, used for help text and eventual UI
    virtual AZStd::string_view GetDescription() const = 0;
 
    // evaluate a rule and return true if there is a match for this device attribute
    virtual bool Evaluate(AZStd::string_view rule) const = 0;
 
    // get the value of this attribute
    virtual AZStd::Any GetValue() const = 0;
};
 
class DeviceAttributeRegistrarInterface
{
    // register a device attribute interface, deviceAttribute must be unique, returns true on success   
    virtual bool RegisterDeviceAttribute(AZStd::string_view deviceAttribute,  AZStd::unique_ptr<IDeviceAttributeInterface> deviceAttributeInterface) = 0;
 
    // visit device attribute interfaces with a callback function   
    virtual void VisitDeviceAttributes(const VisitInterfaceCallback&) const = 0;
 
    // find a device attribute interface
    virtual IDeviceAttributeInterface* FindDeviceAttribute(AZStd::string_view deviceAttribute) const = 0;
};
using DeviceAttributeRegistrar = AZ::Interface<DeviceAttributeRegistrarInterface>;

Initial device attributes will be created for:

• Device model
• RAM
• GPU Vendor
• GPU Model
• GPU Memory

Device-specific settings

Rules can be specified in Settings Registry files to set CVARs based on device attributes like the model, amount of RAM etc.
Rules are ECMAScript regular expressions or short LUA script that evaluates to true or false.

If a device matches multiple rules, the system will use the /O3DE/DeviceRulesResolution setting to determine how apply the rules. Possible values are:

• "Min" use the lowest setting for all matching rules
• "Max" use the highest settings for all matching rules
• "First" use the first rule that matches, based on the order of the rules in the Settings Registry
• "Last" use the last rule that matches, based on the order of the rules in the Settings Registry (DEFAULT)

A sys_print_device_rules console command will output all matching rules and their settings to the console to help developers debug device rules.

Warnings will be displayed in the console logs when a device matches multiple rules.

Custom rules can be written in LUA so developers can write rules that are difficult or impossible with regular expressions. LUA rules are enclosed within a dollar sign and open and closed parenthesis '$()'

Example: "apiVersion":"$((value > 1.2 and value < 2.4 ) and value != 1.5)"

Alternately, a more verbose option is to use an object format like:

 "apiVersion":{
     "Lua": "(value > 1.2 and value < 2.4 ) and value != 1.5"
  }

File: Registry/devices.setreg

{
  "O3DE":{
    "Devices":{
      // settings for the Graphics group for all devices that match rules in the external .setreg file
      "GPU low":{          // A human readable descriptive name for the device group
          "Settings": {
              "q_graphics":0 // the graphics quality level to use
          },
          "Rules":{        // apply the settings for all devices that match any of these rules
              "$import":"devices.gpu-low.rules.setreg"  // import rules from an external file
          }
      },
 
      // example of importing device settings/rules for Samsung devices
      "Samsung": {
         "$import":"devices.samsung.setreg"
      },
 
      // example of custom override rule
      "Custom": {         // A human readable descriptive name for the device group
          "Settings": {   // The name of the scalability group to use with the Level
              "q_graphics":0,       // the Graphics quality level to use
              "r_shadows":[0,0,1,1] // device specific settings overrides
          }
          "Rules":{   // apply the quality level and settings for all devices that match any of these rules
              "LG Devices": {"model":"^LG-H8[235]\\d"}, // regex model match
              "GeForce 6800": {"gpuVendor":"0x10DE", "gpuModel":"0x004[0-8]"}, // gpu vendor and gpu model regex
              "Adreno": {"gpuName":"Adreno.*530", "apiVersion":"Vulcan[123]"}, // gpu name regex with graphics api version regex
              "Experimental":{"$import":"devices.experimental.rules.setreg" } // import rules from an external file
          }
      },
      "LG XYZ Overrides": {
          // this example shows how you would override settings for a specific device match
          // without providing the overall quality level cvar
          "Rules":{
              "LG Devices": {"model":"^LG-H8[235]\\d"}, // regex model match
              "LUA example": {"apiVersion":"$value > 1.0 and value < 2.5"}
          },
          "Settings":{
              "r_shadows":0 // device specific settings override
          }
      }
  }
}

What are the advantages of the feature?

• Meets the requirements
• Uses existing systems (CVARs, Settings Registry, PAL)
• Data-driven settings, groups, rules
• Flexible and customizable

What are the disadvantages of the feature?

• Additional processing of quality settings and device rules on application start up.
• Developers must learn how to use the new system.

How will this be implemented or integrated into the O3DE environment?

The bulk of the implementation is explained in the technical description section. Developers with existing projects will need to copy and customize device rules and quality settings registry files from the default template into their own project and customize them.

Are there any alternatives to this feature?

1. Re-introduce the Lumberyard .cfg spec system.
   1. This system is known to work, but all of the specifications and CVAR setting would need to be updated to work with O3DE.
   2. The primary reason this option was not selected is it relies on hard coded specification detection and does not use the Settings Registry.
   3. Settings for multiple platforms are intermingled instead of using PAL to separate them.
2. Use a simple hard-coded low/medium/high/very-high quality system.
   1. Because O3DE is a modular engine with uses outside of gaming it is unlikely that a "one size fits all" approach will suit the needs of all developers. And making changes would mean the developers would need to modify c++ code and recompile the engine.
3. Port the Lumberyard spec .cfg system and device .xml and .txt files into Settings Registry.
   1. This is a similar amount of work and less flexible to the system that is proposed, the main time/effort savings would be we would only support REGEX for device rules and wouldn't need to have debugging CVARS, but we'd have the limitations of the Lumberyard system.

How will users learn this feature?

Documentation will be available on the main website and in comments inside the Settings Registry files.

Are there any open questions?

1. What is the best name for this framework/system? 'Quality' or 'Scalability'?
2. What are the best locations in the Settings Registry for the system?
3. Is there a better/simpler way to define device rules than using REGEX and LUA?
4. Should this system also let developers specify asset build quality settings?

[moudgils]

Much needed feature finally getting added to O3de :) It will allow the content to be scaled based on device requirements which is critical. Also, I am against going back to LY system as an alternative as it was not as scalable and had other issues around the order in which the cfg files got loaded in memory. The design shown above looks good. My only comment is around debugibility for cvar values. Basically how do we track why a cvar has a value that it has. This will become critical when we want to figure out why a specific rendering feature is enabled or disabled on a specific device when it shouldnt be.

Possible suggestion - Maybe we have a debug menu that shows the regex or Lua expression that was passed to assign a device a specific quality setting and then we could provide the name of the setting registry file that was used to pick up the actual value.

[galibzon]

Much needed feature finally getting added to O3de :) It will allow the content to be scaled based on device requirements which is critical. Also, I am against going back to LY system as an alternative as it was not as scalable and had other issues around the order in which the cfg files got loaded in memory. The design shown above looks good. My only comment is around debugibility for cvar values. Basically how do we track why a cvar has a value that it has. This will become critical when we want to figure out why a specific rendering feature is enabled or disabled on a specific device when it shouldnt be.

Possible suggestion - Maybe we have a debug menu that shows the regex or Lua expression that was passed to assign a device a specific quality setting and then we could provide the name of the setting registry file that was used to pick up the actual value.

IIRC, there was an initiative to being able to visually trace, with a UI, the values of all the keys in the registry. Maybe @lumberyard-employee-dm would know what's the status on that.

[galibzon]

Regarding this open question:

Should this system also let developers specify asset build quality settings?

I think builders would get the ability for "free". Meaning, Asset Builders can also read the content of the settings registry and read the quality/scalability values.

[moudgils]

Should this system also let developers specify asset build quality settings?

This is a much more complex problem and has significant repercussions. Remember we only have one asset folder per platform and not a folder per platform and per quality setting tier. So if we want to for example build the same asset with different compressions we will need to modify our asset building code to provide support for that at asset processing time, asset packaging time as well as as asset loading time when running the app. My recommendation is to not consider different tiers of assets at this time but keep it in mind for future. Building a general system would be harder to do but the best way forward. If that becomes too large of an effort you could revisit this problem per asset type and implement a custom solution (which is not ideal).

[lemonade-dm]

Much needed feature finally getting added to O3de :) It will allow the content to be scaled based on device requirements which is critical. Also, I am against going back to LY system as an alternative as it was not as scalable and had other issues around the order in which the cfg files got loaded in memory. The design shown above looks good. My only comment is around debugibility for cvar values. Basically how do we track why a cvar has a value that it has. This will become critical when we want to figure out why a specific rendering feature is enabled or disabled on a specific device when it shouldnt be.
Possible suggestion - Maybe we have a debug menu that shows the regex or Lua expression that was passed to assign a device a specific quality setting and then we could provide the name of the setting registry file that was used to pick up the actual value.

IIRC, there was an initiative to being able to visually trace, with a UI, the values of all the keys in the registry. Maybe @lumberyard-employee-dm would know what's the status on that.

We have the backend in place to traces the origin of all loades are coming from with the Settings Registry origin tracker.

There is also a still functioning prototype of the SettingsRegistry Editor that is available in the DPE Debug View Stadnalone application: o3de/o3de#11404

But no work has been done to integrate the Settings Registry Editor with the Editor and properly Gemify it.

[AMZN-alexpete]

we only have one asset folder per platform and not a folder per platform and per quality setting tier

There is also the possibility of user-defined asset processor platforms like android_low, android_high to build assets in different ways, but then you couldn't switch them at runtime to preview the different assets in the editor - we'd also need to consider if asset seed lists would work so the release packaging is correct. It sounds complicated enough that I'm leaning toward it being a separate RFC.

[AMZN-alexpete]

The design shown above looks good. My only comment is around debugibility for cvar values. Basically how do we track why a cvar has a value that it has. This will become critical when we want to figure out why a specific rendering feature is enabled or disabled on a specific device when it shouldnt be.

Possible suggestion - Maybe we have a debug menu that shows the regex or Lua expression that was passed to assign a device a specific quality setting and then we could provide the name of the setting registry file that was used to pick up the actual value.

Because the design uses CVars we should be able to always see the values of the group quality level and the actual cvars from the console, and for debugging device rules having additional console commands like sys_print_device_rules can be created to provide minimal text only debug output till we tackle the UI. Maybe having one like sys_trace_device_rule <cvar> and it could output the device rules and .setreg file where the setting came from for that cvar. Something like:

sys_trace_device_rule r_shadow_quality
using r_shadow_quality : 3 from "registry/quality_android.setreg" line 100
based on device rule : "model":"MD[0-9]"  in "registry/quality_android.setreg" line 80
additional device rules with lower precedence:
"gpuVendor":"ABC" in "Gems/Atom/Registry/quality_general.setreg" line 79
...

[lemonade-dm]

Since the levels can be mapped to enumerations in C++ code, I think it is better to avoid having spaces in the level.
Plus using a lowercase second word opens up the possibility of string comparison mismatch due to case issues.
In the examples above it shows using "Very high" as a level.

{
  "O3DE": {
    "Quality": {
      "q_graphics": { // General Graphics settings that uses shadow and visibility quality group settings
        "Levels": [ "Low", "Medium", "High", "Very high" ],

Also for the DeviceAttribute inteface instead of exposing an AZStd::any as a value, we can stringify the value using JSON Serializer or the Settings Registry DumpSettingsToStream function.

I also recommend removing the m_deviceAttribute AZStd::string from the interface and let the derived class determine the string type they would like to use for the Attribute name(i.e QString, AZStd::fixed_string, const char*, AZStd::string, etc...)

struct IDeviceAttributeInterface
{
    // get the name of the device attribute e.g. gpuMemory, gpuVendor, customAttribute42
    virtual AZStd::string_view GetDeviceAttribute() const { return m_deviceAttribute; }
     
    // get a description about this device attribute, used for help text and eventual UI
    virtual AZStd::string_view GetDescription() const = 0;
 
    // evaluate a rule and return true if there is a match for this device attribute
    virtual bool Evaluate(AZStd::string_view rule) const = 0;
 
    // get the value of this attribute
    virtual AZStd::string GetValue() const = 0;

};

For how Lua is being used, I don't like the syntax for it.
"LUA example": {"apiVersion":"$value > 1.0 and value < 2.5"}

Looking at the example, I think it is easy for a user familiar with bash, powershell, perl or php to get confused.
$value looks like a variable reference, while the second value later on looks out of place.
How about using the bash command substitution approach of $(command) which uses parenthesis as a delimiter

Therefore the example above becomes
"LUA example": {"apiVersion":"$(value > 1.0 and value < 2.5)"}

[AMZN-alexpete]

Since the levels can be mapped to enumerations in C++ code, I think it is better to avoid having spaces in the level.

That's probably best, initially I was thinking it'd be great for users to be able to directly use the text version in their UI if desired, but in most games you'll need to translate to the correct language so it might as well be a single word.

I also recommend removing the m_deviceAttribute AZStd::string from the interface and let the derived class determine the string type

Agreed! I came to the same conclusion when prototyping this.

How about using the bash command substitution approach of $(command) which uses parenthesis as a delimiter

I'd be OK with that. And another option we discussed would be to use an Object format like:

{
    "Lua example":{
        "apiVersion":{
            "Lua":"value > 1.0 and value < 2.5"
        }
    }
}

or

{
    "LUA example":{
        "apiVersion":{
            "type":"Lua"
            "rule":"value > 1.0 and value < 2.5"
        }
    }
}

[lemonade-dm]

As a representative of @o3de/sig-core we are signing off on this RFC for implementing the proposed framework for configuring quality levels and scalability logic into O3DE.

There are some logistical information I would like added in the RFC, such as the flavor of Regex supported(Basic, Extended, ECMAScript). It most likely is ECMAScript since that is default value for AZStd::regex.

It could also be mentioned if level of character escaping that is needed for device rules since they will be written in JSON.
For example are 4 backslashes needed to get match a literal backslash or 2 (\\\\ vs \\).

Finally you may want to have a mention that all strings that don't trigger Lua execution are treated as Regular Expressions and never plain text. This is so that users know that if don't want to match a dollar sign, it would need to be escaped.