Consider including hello-world example eval

[MattFisher]

When getting started with k8s and Inspect Evals, it can be difficult to debug the basics.

I was trying to get inspect_evals/swebench working with k8s but that had lots of extra complexity, so I wrote a small test task that might be a useful inclusion for others going through the same thing.

# k8s_task.py

from inspect_ai import Task, task
from inspect_ai.dataset import MemoryDataset, Sample
from inspect_ai.scorer import includes
from inspect_ai.solver import generate, use_tools
from inspect_ai.tool import bash


@task
def hello_k8s() -> Task:
    samples = [
        Sample(
            input="Get the OS version codename using `cat /etc/os-release`.",
            target="bookworm",
        ),
        Sample(
            input="Get info on the web server version running at my-web-server.com.",
            target="nginx/1.27.0",
        ),
    ]
    return Task(
        dataset=MemoryDataset(samples=samples),
        solver=[
            use_tools([bash()]),
            generate(),
        ],
        sandbox=("k8s", "values.yaml"),
        scorer=includes(),
    )

# values.yaml

services:
  default:
    image: python:3.12-bookworm
    command: ["tail", "-f", "/dev/null"]
  server:
    image: nginx:1.27.0
    dnsRecord: true
    additionalDnsRecords:
      - "my-web-server.com"
    readinessProbe:
      tcpSocket:
        port: 80

[craigwalton-dsit]

I think this is a great idea. I often end up copying one from the tests, but I appreciate this is not very discoverable. I'm a fan of Inspect's examples dir - I often copy the hello_world.py example from it.

[tcatling]

Are you happy for me to close this? I think this example has since been added