Support partial success of test (for aggregate tests)

[fuhrmanator]

I'm using this with npm test which runs a npx jest command that runs many tests. It's not useful to me to have a max-score: 100 that results in either 0 points or 100 points, with no partial (percentage) value for partial test success. For reference, see classroom-resources/autograding-example-node#4

It would be interesting if there was an option for partial success that somehow scaled the points of the result based on the number of successful (inside the aggregate) tests that pass.

Each testing environment is different, but maybe a test command (maybe a special script) can pass some environment variables for the number of tests that pass/fail and the action can calculate the points accordingly.

Otherwise, I don't see this action getting much uptake (at least in my courses) where multiple tests are defined inside a project using standard harnesses. The CoPilot script I proposed in the issue I cited above works for Jest, but it generates a static YAML file (dynamically, based on the jest configuration) as a work-around (it still invokes the action once for each test file). It's still not perfect, since each test file (suite) inside a project can have multiple tests, and the action results in either all the points (if all tests pass) or none (if at least one test fails), which is still not ideal.

[piti118]

I workaround this by having it making multiple tests each run a specific set of tests.

Ex:

q1: --> go test hw/q1 --> 10 point (yep it's still 0 or 10)
q2: --> go test hw/q2 --> 10 point

name: Autograding Tests
'on':
- push
- repository_dispatch
permissions:
  checks: write
  actions: read
  contents: read
jobs:
  run-autograding-tests:
    runs-on: ubuntu-latest
    if: github.actor != 'github-classroom[bot]'
    steps:
    - name: Checkout code
      uses: actions/checkout@v4
    - name: q1
      id: q1
      uses: classroom-resources/autograding-command-grader@v1
      with:
        test-name: q1
        setup-command: ''
        command: go test go-beginner/q1
        timeout: 10
        max-score: 10
    - name: q2
      id: q2
      uses: classroom-resources/autograding-command-grader@v1
      with:
        test-name: q2
        setup-command: ''
        command: go test go-beginner/q2
        timeout: 10
        max-score: 10
    - name: Autograding Reporter
      uses: classroom-resources/autograding-grading-reporter@v1
      env:
        Q1_RESULTS: "${{steps.q1.outputs.result}}"
        Q2_RESULTS: "${{steps.q2.outputs.result}}"
      with:
        runners: q1,q2