is it possible to work with multiple {lint} results in CI/CD

[Josverl]

I would like to be able to report from a github action on different linting result sets in testspace.

I tried to add multiple resultsets , but the effect was that none of them were recognized.

      - name: Testspace push test content
        run: |
          testspace "[all-stubs]./results/pyright-all-stubs.log{lint}" ./results/results.xml
          testspace "[v1_18]./results/pyright-v1_18.log{lint}"
          testspace "[latest]./results/pyright-latest.log{lint}"

looking at the upload this appears to work but the results are all empty.
is this possible, and if so how ?

[ivailop]

Yes, it is possible and should be working as expected.

I think the issue is on your side - you specified wrong slashes (\ vs /) at https://github.com/Josverl/micropython-stubs/blob/0212fb42caea9b9934a4185344d6d1c822044bc6/.github/workflows/testspace.yml#L47-L48. As a result of that the more specific pyright log files end up empty.

[Josverl]

Ah , but an error that was already corrected before i posted my question:

below is the commit from which I copied the initial question, with all bash style slashes

Josverl/micropython-stubs@b874985#diff-63ef1968d2d361e0f886193c032aa7e8361c1d1defb593ad84a87e85748d79e6

[ivailop]

There is no problem with the testspace command lines (that you posted initially).
The slashes (\ vs /) issue I pointed out is with the pyright command lines (in the 'run pyright on stubs" job) - pyright stubs\micropython... should be pyright stubs/micropython...

[Josverl]

🙏🏼Thanks for pointing out the issue.
that indeed allows me to track the separate subsets :-)

Now new questions come to mind such as

• how to compare the relative ratings ( Last versus Stable ) without needing to send all logs multiple times
  I suppose running a script to do that sort of comparison and push it though a CSV could work. is that the only way ?
• How to compare a trend ( Last versus Last-1 ) and preferably push that back that trend info to the commit or PR
  I think this already works for Tests, but in this case the 'test' is actually the quality of the static analysis

[ivailop]

Now new questions come to mind such as

• how to compare the relative ratings ( Last versus Stable ) without needing to send all logs multiple times
  I suppose running a script to do that sort of comparison and push it though a CSV could work. is that the only way ?
• How to compare a trend ( Last versus Last-1 ) and preferably push that back that trend info to the commit or PR
  I think this already works for Tests, but in this case the 'test' is actually the quality of the static analysis

Sorry for the late replay... Those is good questions (feature requests) however they are (IMHO) outside of the Testspace scope.
Collection of "Static Analysis" (and other) metrics is supported in Testspace for the purpose of aiding (as auxiliary piece of data) the primary test results.

[Josverl]

OK, a pity, but I understand the need for focus.