Replacing OK&FAIL with MET&UNMET for UseCase Conditions

[ArnoldTchegou]

Hi,

Currently, when the UseCasecomparison is evaluated as false, it appears as FAIL in the output HTML page, and when evaluated as true, it appears as OK.

To my understanding, a usecase condition should not necessarily be considered as FAIL as this term is typically associated with requirements themselves.

Instead, a usecase condition serves as a prerequisite for applying or not applying underlying requirements, rather than being a requirement in itself.

I propose that the results field in the HTML page be adjusted to reflect this distinction:

• If the condition evaluates to true, it should be shown as MET rather than OK.
• If the condition evaluates to false, it should be shown as UNMET.

If you are willing to welcome these changes, then i will start the implementation for a future pull request.

[mraineri]

Do you have a sample profile showing what you're testing?

One thing that may not be clear from the spec is that use cases are mandatory (as in at least one resource matches the use case in the profile) unless "ReadRequirement": "Recommended" is added in the use case itself. I'm wondering if you're hitting this case.

[ArnoldTchegou]

Hello Mike,

You are correct that a Usecase is mandatory by default.

However, I have some concerns about the validator’s current behavior when handling UsecaseComparison values of AllOf, AnyOf, or Equal.

Currently, the validator first checks if there is/are any resources(s) that satisfy the Usecase condition, and then proceeds to validate the Usecase Readrequirement.

For example, consider the following ResourceUsecase on Sensor:

"SchemaDefinition": "RedfishInteroperabilityProfile.v1_8_0",
"ProfileName": "sensor_v1",
"ProfileVersion": "1.0.0",
"Purpose": "Service must implement Temperature, Power sensors",
"Resources": {
    "Sensor": {
        "UseCases": [
            {
                "UseCaseTitle": "UsecaseReadingType",
                "UseCaseKeyProperty": "ReadingType",
                "UseCaseComparison": "<AllOf || AnyOf || Equal>",
                "UseCaseKeyValues": [
                    "Temperature",
                    "Power"
                ],
                "PropertyRequirements": {
                    
                }
            }
        ]
    }
}

The table below summarizes how the validator currently evaluates each UseCaseComparison (for Mandatory UseCase):

Resource(s) Exist	Current AllOf	Expected AllOf	Current AnyOf	Expected AnyOf	Current Equal	Expected Equal
YES	FAIL	OK	OK	OK	OK	OK
NO	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL

As shown above, the case in bold (AllOf) needs to be addressed. In this scenario, the service correctly implements the Usecase condition for AllOf, but the validator still marks it as FAIL.

I have started working on a fix for this. If you agree that these changes are worthwhile, I can submit a pull request.

[mraineri]

The intent of "UseCaseComparison" is to be a decision point for when to step into the use case for further testing. We did not envision using AnyOf or AllOf at all for this (even though we are leveraging the same comparison enumeration for possible values). For Equal, the intent is the use case is mapped when the UseCaseKeyProperty contains one of the values listed in UseCaseKeyValues.

Do you have a need to use AnyOf or AllOf for this? Are you able to provide some details behind the profile you're constructing now? Perhaps there's a more simplistic way we can recommend crafting the profile.