feat: external media references — store large media by typed payload URI (#115)

[dcfocus]

Summary

Adds a typed external-reference payload so large media (images/audio/video) can live as objects in the configured object store and be referenced from a ContextRecord by payload_uri (plus optional payload_size / payload_checksum), instead of inlining bytes in binary_payload.

• list / search return the reference without materializing the bytes.
• An opt-in fetch resolves the bytes on demand using the context's storage_options — works for gs://, s3://, and local paths through the same object-store path the dataset itself uses.
• Inline binary_payload stays the unchanged small-payload path.

Closes #115

What changed

core (lance-context-core)

• payload_uri / payload_size / payload_checksum on ContextRecord and RecordPatch.
• New schema columns gated by an include_external_reference flag (mirrors the existing external_id / metadata / lifecycle gates) → backward compatible: old datasets read them as None; writing a reference on a dataset created without the columns errors clearly.
• ContextStore::fetch_payload(id) resolves payload_uri → bytes, and ContextStore::put_payload(uri, bytes) offloads caller bytes, both via ObjectStore::from_uri_and_params threading the context's storage_options.

api / server / client / python

• Fields threaded through AddRecordRequest / RecordDto / RecordPatchDto (all Option, skip_serializing_if → serde back-compat).
• GET /api/v1/contexts/{name}/records/{id}/payload (200 octet-stream / content_type, 404 missing record, 400 record with no reference).
• Rust client fetch_record_payload; Python Context.fetch_payload / put_payload.

Tests

• Rust: add→list (ref present, binary_payload None)→fetch_payload roundtrip; missing-record (None) and missing-reference (error) cases; put_payload roundtrip; server route test (bytes / 404 / 400); api serde round-trip + back-compat decode.
• Python: end-to-end put_payload → add(payload_uri=…) → list/get (no bytes) → fetch_payload; update-attaches-reference-later; missing-record/missing-reference.

Notes for the reviewer

• MemWAL visibility: fetch_payload (and the server handler) use the list-backed get_by_id rather than the raw filtered get(id), because the latter does not see freshly-written MemWAL-buffered rows in the same process.
• Pre-existing test doubles realigned: the shared add/upsert/update doubles in test_search.py / test_embeddings.py were already out of sync with the current binding signature (missing tenant/source/run_id/created_at) and failing on main. Since this PR extends those exact signatures, the doubles are brought back in sync and extended for the new payload params. (test_persistence.py::test_s3_* remain env-dependent and are unaffected.)
• Out of scope: signed-URL resolution is deferred — a TODO marks where a presigned-URL branch would go. No transcoding/thumbnailing, no media embedding, inline binary_payload unchanged.

🤖 Generated with Claude Code