"Semantic" markup in SARIF messages

[davidmalcolm]

There doesn't seem to be a way to add "semantic" markup to messages in SARIF logs.

For example, in GCC 15 I've added colorization to some diagnostics to highlight expected type vs actual type; see:
https://godbolt.org/z/zqWGe5jKh

where note how

• the %i in the message line and quoted source and the int in the message line and the source annotation all appear in green (they're marked up internally as expected_type)
• the const char * in the message line and in the source annotation and the msg in the quoted source all appear in dark blue (they're marked up internally as actual_type)

I'd like to capture this markup in the generated SARIF output, but we only support a subset of Markdown, and, in particular, HTML is forbidden.

[KalleOlaviNiemitalo]

What does GCC output for that in SARIF now, without colorization?

My first thought is that I'd place the color annotations in the multiFormatMessageString object in the reportingDescriptor and associate them with the placeholder numbers, something like:

{
    "text": "format '{0}' expects argument of type '{1}', but argument {2} has type '{3}' [{4}]",
    "placeholders": {
        "0": { "role": "expected_type" },
        "1": { "role": "expected_type" },
        "3": { "role": "actual_type" },
        "4": { "role": "option" }
    }
}

or as an array:

{
    "text": "format '{0}' expects argument of type '{1}', but argument {2} has type '{3}' [{4}]",
    "placeholders": [
        { "role": "expected_type" },
        { "role": "expected_type" },
        {},
        { "role": "actual_type" },
        { "role": "option" }
    ]
}

This scheme would make the color information trivial to ignore for SARIF consumers that don't care about it. However, this does not seem easy to apply to msg in the quoted source code, and to the diagram below it.

[davidmalcolm]

What does GCC output for that in SARIF now, without colorization?

From https://godbolt.org/z/9ja73xv19:

  "results": [{"ruleId": "-Wformat=",
                        "level": "warning",
                        "message": {"text": "format '%i' expects argument of type 'int', but argument 2 has type 'const char *'"},
                        "locations": [{"physicalLocation": {"artifactLocation": {"uri": "<source>"},
                                                            "region": {"startLine": 5,
                                                                       "startColumn": 17,
                                                                       "endColumn": 19},
                                                            "contextRegion": {"startLine": 5,
                                                                              "snippet": {"text": "  printf(\"hello %i\", msg);\n"}}},
                                       "logicalLocations": [{"name": "test_mismatching_types",
                                                             "fullyQualifiedName": "test_mismatching_types",
                                                             "decoratedName": "test_mismatching_types",
                                                             "kind": "function"}],
                                       "annotations": [{"startLine": 5,
                                                        "startColumn": 17,
                                                        "endColumn": 19,
                                                        "message": {"text": "int"}},
                                                       {"startLine": 5,
                                                        "startColumn": 22,
                                                        "endColumn": 25,
                                                        "message": {"text": "const char *"}}]}],
                        "fixes": [{"artifactChanges": [{"artifactLocation": {"uri": "<source>"},
                                                        "replacements": [{"deletedRegion": {"startLine": 5,
                                                                                            "startColumn": 17,
                                                                                            "endColumn": 19},
                                                                          "insertedContent": {"text": "%s"}}]}]}]}]}]}

[davidmalcolm]

What does GCC output for that in SARIF now, without colorization?

From https://godbolt.org/z/9ja73xv19:

Using placeholders, this could be:

  "results": [{"ruleId": "-Wformat=",
                        "level": "warning",
                        "message": {"text": "format '{}' expects argument of type '{}', but argument {} has type '{}'",
                                    "arguments": ["%i",
				                  "int",
						  "2",
						  "const char *"],
				    "roles": {"0": "expected-type",
				              "1": "expected-type",
					      "3": "actual-type"}},
                        "locations": [{"physicalLocation": {"artifactLocation": {"uri": "<source>"},
                                                            "region": {"startLine": 5,
                                                                       "startColumn": 17,
                                                                       "endColumn": 19},
                                                            "contextRegion": {"startLine": 5,
                                                                              "snippet": {"text": "  printf(\"hello %i\", msg);\n"}}},
                                       "logicalLocations": [{"name": "test_mismatching_types",
                                                             "fullyQualifiedName": "test_mismatching_types",
                                                             "decoratedName": "test_mismatching_types",
                                                             "kind": "function"}],
                                       "annotations": [{"startLine": 5,
                                                        "startColumn": 17,
                                                        "endColumn": 19,
                                                        "message": {"text": "int"},
							"role": "expected-type"},
                                                       {"startLine": 5,
                                                        "startColumn": 22,
                                                        "endColumn": 25,
                                                        "message": {"text": "const char *"}},
							"role: "actual-type"]}],
                        "fixes": [{"artifactChanges": [{"artifactLocation": {"uri": "<source>"},
                                                        "replacements": [{"deletedRegion": {"startLine": 5,
                                                                                            "startColumn": 17,
                                                                                            "endColumn": 19},
                                                                          "insertedContent": {"text": "%s"}}]}]}]}]}]}

where I've added a "roles" property to the "message", and "role" properties to the annotations - though semantically we could draw a distinction between "%i" and "int" here.

[davidmalcolm]

I have an work-in-progress implementation of GCC's SARIF output that can emit placeholders when generating message objects. FWIW it currently emits this for the above example:

"message": {"text": "{0} {1} expects argument of type {2}, but argument {3} has type {4}",
            "arguments": ["format",
                          "‘%i’",
                          "‘int’",
                          "2",
                          "‘const char *’"]},

I'm looking at implementing a prototype of the ideas from the above comments.

[davidmalcolm]

I have a working prototype of the idea from #662 (comment)

The current output on the example is:

"message": {"text": "{0} {1} expects argument of type {2}, but argument {3} has type {4}",
            "arguments": ["format",
                          "‘%i’",
                          "‘int’",
                          "2",
                          "‘const char *’"],
            "roles": {"1": "highlight-a",
                      "2": "highlight-a",
                      "4": "highlight-b"}},
"locations": [{"physicalLocation": {"artifactLocation": {"uri": "../../src/gcc/testsuite/gcc.dg/format/diagnostic-ranges.c",
                                                         "uriBaseId": "PWD"},
                                    "region": {"startLine": 9,
                                               "startColumn": 17,
                                               "endColumn": 19},
                                    "contextRegion": {"startLine": 9,
                                                      "snippet": {"text": "  printf(\"hello %i\", msg);\n"}}},
               "logicalLocations": [{"name": "test_mismatching_types",
                                     "fullyQualifiedName": "test_mismatching_types",
                                     "decoratedName": "test_mismatching_types",
                                     "kind": "function"}],
               "annotations": [{"startLine": 9,
                                "startColumn": 17,
                                "endColumn": 19,
                                "message": {"text": "int"},
                                "role": "highlight-a"},
                               {"startLine": 9,
                                "startColumn": 22,
                                "endColumn": 25,
                                "message": {"text": "const char *"},
                                "role": "highlight-b"}]}],

adding a roles object to the message, and role properties to the annotations.

Couple of issues here:

• the "roles" here are presentation-based, rather than semantic (the former is what I'm currently capturing inside GCC's diagnostics subsystem; I could add the latter and use them)
• I'm just using the first role found in the token stream; there's no way to express semantics with a granularity finer than individual placeholder arguments. Not sure if that's a real limitation though.

[davidmalcolm]

I prototyped doing it with property bags instead:

"message": {"text": "{0} {1} expects argument of type {2}, but argument {3} has type {4}",
            "arguments": ["format",
                          "‘%i’",
                          "‘int’",
                          "2",
                          "‘const char *’"],
            "properties": {"gcc/argumentPresentation": {"1": "highlight-a",
                                                        "2": "highlight-a",
                                                        "4": "highlight-b"}}},
"locations": [{"physicalLocation": {"artifactLocation": {"uri": "../../src/gcc/testsuite/gcc.dg/format/diagnostic-ranges.c",
                                                         "uriBaseId": "PWD"},
                                    "region": {"startLine": 9,
                                               "startColumn": 17,
                                               "endColumn": 19},
                                    "contextRegion": {"startLine": 9,
                                                      "snippet": {"text": "  printf(\"hello %i\", msg);\n"}}},
               "logicalLocations": [{"name": "test_mismatching_types",
                                     "fullyQualifiedName": "test_mismatching_types",
                                     "decoratedName": "test_mismatching_types",
                                     "kind": "function"}],
               "annotations": [{"startLine": 9,
                                "startColumn": 17,
                                "endColumn": 19,
                                "message": {"text": "int"},
                                "properties": {"gcc/presentation": "highlight-a"}},
                               {"startLine": 9,
                                "startColumn": 22,
                                "endColumn": 25,
                                "message": {"text": "const char *"},
                                "properties": {"gcc/presentation": "highlight-b"}}]}],

where the message has gained a property bag with a gcc/argumentPresentation map and each entry in annotations has gained a gcc/presentation value.

[davidmalcolm]

* I'm just using the first role found in the token stream; there's no way to express semantics with a granularity finer than individual placeholder arguments.  Not sure if that's a real limitation though.

A concrete example where this is a problem to round-tripping:

where the SARIF has:

"message": {"text": "invalid operands to binary {0} (have {1} and {2})",
            "arguments": ["+",
                          "‘S’ {aka ‘struct s’}",
                          "‘T’ {aka ‘struct t’}"],
            "properties": {"gcc/argumentPresentation": {"1": "highlight-a",
                                                        "2": "highlight-b"}}},

but in the screenshot above note how the colorization of {1} corresponds to:

‘<highlight-a>S</highlight-a>’ {aka ‘<highlight-a>struct s</highlight-a>’}

which contains internal structure within the arguments.

(bother)

[sthagen]

Question on:

[...] the "roles" here are presentation-based, rather than semantic (the former is what I'm currently capturing inside GCC's diagnostics subsystem; I could add the latter and use them) [...]
-- @davidmalcolm

Curiosity k.t. 🐈: How would semantic roles in this case look like?