Skip to content

feat: Cost-aware format conversion via zenpixels-convert negotiate #5

@lilith

Description

@lilith

Summary

ensure_format() in graph.rs:1691-1707 is naive — it checks if current format equals target format, and if not, unconditionally inserts a RowConverterOp. No cost estimation, no alternative path consideration, no quality loss tracking.

Meanwhile, zenpixels-convert already has a sophisticated cost model and path solver that zenpipe depends on but doesn't use for planning:

  • Two-axis cost model: effort (CPU work, 0-255) + loss (quality destroyed, percentage/bits)
  • ConvertIntent weighting: Fastest (4× effort, 1× loss), LinearLight (1× effort, 4× loss), Blend, Perceptual
  • Three-tier path finding: direct kernels → composed multi-step → hub path (via linear sRGB f32)
  • Quality thresholds: Lossless, SubPerceptual (ΔE<0.5), NearLossless (ΔE<2.0), MaxBucket
  • Provenance tracking: knows f32-from-u8-JPEG is lossless roundtrip
  • negotiate(): picks best format from candidates weighted by intent

Current behavior

fn ensure_format(source, target) -> Result<Box<dyn Source>> {
    if current == target { return Ok(source); }         // match → no-op
    let op = RowConverterOp::new(current, target)?;     // mismatch → insert unconditionally
    Ok(Box::new(TransformSource::new(source).push(op)))
}

Problems:

  1. No cost awareness — RGBA8_SRGB→RGBAF32_LINEAR is expensive but always inserted
  2. No alternative paths — doesn't consider keeping data in a compatible intermediate format
  3. No quality loss tracking — can't warn when conversion chain destroys quality
  4. Hardcoded format targets — Layout forces RGBA8_SRGB (line 1134), Resize forces RGBA8_SRGB (line 1207), Composite forces RGBAF32_LINEAR_PREMUL (lines 1335-1336)

Proposed changes

  1. Replace format forcing with negotiation — instead of hardcoding RGBA8_SRGB, collect each operation's acceptable formats and call zenpixels_convert::negotiate() to pick the cheapest compatible format
  2. Track cumulative quality loss — sum loss scores through conversion chain; warn in trace when exceeding threshold
  3. Avoid redundant round-trips — if source is already f32 linear and next op wants f32 linear, don't convert to u8 sRGB in between
  4. Use operation format preferences — let each NodeOp declare its preferred/acceptable formats via OpCategory (zenpixels-convert already has this)
  5. Expose cost decisions in tracing — the tracer already records implicit conversions with reasons; add cost scores

What zenpixels-convert provides

  • zenpixels-convert/src/negotiate.rsnegotiate(), best_match(), ConversionCost, ConvertIntent
  • zenpixels-convert/src/pipeline/path.rsoptimal_path(), ConversionPath, QualityThreshold
  • zenpixels-convert/src/pipeline/op_format.rsOpCategory requirements and candidate generation
  • zenpixels-convert/src/convert.rsConvertPlan with composed multi-step conversions

What zenimage had (reference only)

zenimage's pipeline/planner.rs + conversion_registry.rs + cost.rs (~2900 lines) added Dijkstra graph search and operation-level integration on top of similar concepts. The Dijkstra approach may be overkill since zenpixels-convert's tiered path finding (direct → composed → hub) handles most cases. The main value from zenimage's planner is the pipeline-level integration: inserting conversions between operations in a multi-op chain, which is what zenpipe needs.

Priority

Medium — correctness is fine today (conversions work), but unnecessary round-trips waste CPU and can degrade quality for HDR/wide-gamut content where precision matters.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions