Score(value=None) is accepted but crashes in value_to_float, so a scorer cannot abstain

[VihanAggarwal]

Summary

Score(value=None) is accepted by the constructor but crashes during scoring, so a scorer that legitimately abstains has no supported way to express that. The only workaround is to emit 0.0, which is indistinguishable from a genuine failure verdict in every log and summary.

This matters for real-robot benchmarks specifically: an operator grader on an unattended run has no verdict, which is a different fact from "the policy failed".

Repro

from dataclasses import dataclass
from inspect_robots import Score, Task, eval
from inspect_robots.scene import Scene
from inspect_robots.registry import resolve

@dataclass(frozen=True)
class Abstain:
    name: str = "abstains"
    def __call__(self, record, target):
        return Score(value=None, explanation="abstain: no operator verdict recorded")

task = Task(name="repro-abstain",
            scenes=[Scene(id="s0", instruction="reach the cube", init_seed=0)],
            scorer=Abstain(), max_steps=20)
eval(task, resolve("policy", "scripted"), resolve("embodiment", "cubepick"), seed=0)

File "inspect_robots/eval.py", line 610, in _run_eval
    epoch_values[scorer.name] = value_to_float(score.value)
File "inspect_robots/scorer.py", line 47, in value_to_float
    return float(value)
TypeError: float() argument must be a string or a real number, not 'NoneType'

Score(value=None, explanation="...") itself constructs without error, so nothing signals that None is unsupported until the rollout has already been spent.

Why this looks unintended rather than by design

cli.py:2029 (the inspect subcommand) already formats a None metric as n/a:

print(f"  {name}: {'n/a' if value is None else f'{value:.4g}'}")

so None-valued metrics appear to be anticipated at the reporting layer. The two other formatters do not guard: cli.py:1377 (the run summary) and cli.py:1846 both use a bare f"{value:.4g}" and would raise TypeError on the same value.

Two directions, and I did not want to presume which

1. Reject early. Validate in Score.__post_init__ so Score(value=None) fails at construction with a clear message. Minimal, but it forecloses abstention and leaves the n/a guard at cli.py:2029 unreachable from scorers.
2. Support abstention end to end. value_to_float returns None, reducers skip abstained epochs rather than counting them as zero, metrics may be None, and the two unguarded formatters adopt the cli.py:2029 idiom. This preserves the distinction between "no verdict" and "verdict: fail" in the logs, which is the property that makes an unattended run auditable after the fact.

Happy to send a PR for either once you have a preference. Direction 2 is the one we would use, but it changes reducer semantics, so it seemed like your call rather than ours.

Environment

inspect-robots 0.58.0 (PyPI), Python 3.13.14, Windows. Reproduced against the cubepick mock embodiment, so no hardware is required.