“Rewrite-then-Edit” two-stage pipelines & system-prompt leaks: do they systematically boost RISEBench scores?

[wyhlovecpp]

Congratulation for Neurips Oral work. But I still have some question. Several models on your leaderboard (e.g., Gemini-2.5-Flash-Image, Seedream, BAGEL w/ CoT) appear to run a two-stage pipeline: an MLLM first rewrites/structures the edit instruction (often with CoT), then a second stage performs the actual image edit. The community has also shown system-prompt leaks from some closed models suggesting this “rewrite-then-generate” template like Seedream and nano banana.

Questions / requests for clarification:

1. If we apply a strong, neutral MLLM rewriter (e.g., GPT-5 / Gemini / Qwen-VL) to all RISEBench prompts before feeding them to open-source editors (e.g., FLUX.1-Kontext-dev, Qwen-Image-Edit), do scores jump substantially?
2. Does the leaderboard distinguish direct instruction vs rewrite-then-edit settings? Could you provide a reproducible toggle and a sub-leaderboard to avoid methodological apples-to-oranges?
3. Would you consider an official Prompt-Rewrite Protocol (with a fixed rewriter + template) so we can attribute gains between “instruction enhancement” and the “editor’s intrinsic capability”?

Thanks—clear guidance here would be helpful

[PhoenixZ810]

Thank you for your interest and thoughtful questions!

• Regarding your first question, the answer is No — this is because a significant portion of samples in RISEBench cannot be easily or concisely described in natural language. This is particularly true for tasks in the Spatial and Logical categories.
• For your second question, we will consider adding a tag to distinguish between end-to-end and multi-stage models.
• As for the third point, we have not yet considered implementing an official Prompt-Rewrite Protocol, as we believe it could introduce unfairness into the evaluation process.

Thank you again for your insightful suggestions, they are greatly appreciated!

[wyhlovecpp]

Hi — thanks for maintaining RISEBench. @PhoenixZ810 @ZpyWHU

I’d like to pair my results with your leaderboard for a fair comparison. I will upload all CSVs (per-run and per-sample) to a Hugging Face repo and can share the link. Before I do that, could you confirm the expected format and a few details so I can match your leaderboard exactly?

Specifically:
1. Please confirm the exact metric/column names and units you use (e.g. Score-Origin, Score-Percentage, Accuracy) and whether any normalization was applied.
2. Do you provide per-sample score files (sample id → metric scores)? If so, could you share an example (or the required schema)?
3. Do you distinguish “end-to-end” vs “rewrite-then-edit” runs on the leaderboard (or provide a toggle)? If not, please indicate how you’d prefer these to be labeled when I upload (e.g. pipeline_type: end-to-end | rewrite-then-edit, rewriter_name, system_prompt_leak: true/false).

My current overall summary (I will attach the full CSVs):
• Overall Score-Origin: 3.2592 (Score-Percentage: 56.48)
• Strong: Visual plausibility / temporal & causal / spatial consistency.
• Weak: Logical reasoning (Logical_Reasoning ≈ 1.42; logic subitems very low).

I can upload a HF dataset containing mine.csv (summary), per_sample.csv, and metadata.json. If you provide the schema you prefer, I’ll match it and post the HF link here. Thanks.

The below is a version about rewrite prompt using by our model.

-,Score-Origin,Score-Percentage,Accuracy
Overall,3.259166666666667,56.47916666666668,0.19166666666666668
Temporal,3.6164705882352943,65.41176470588236,0.16470588235294117
Causal,3.988888888888889,74.72222222222223,0.24444444444444444
Spatial,3.8459999999999996,71.14999999999999,0.26
Logical,1.4388235294117644,10.970588235294109,0.08235294117647059
Overall_Reasoning,3.172222222222222,54.30555555555555,
Overall_ApprConsistency,4.048158640226629,76.20396600566572,
Overall_VisualPlausibility_total,4.541818181818182,88.54545454545455,
Temporal_Reasoning,3.541176470588235,63.52941176470588,
Temporal_Consistency,4.5875,89.68750000000001,
Temporal_Quality,4.3882352941176475,84.70588235294119,
Causal_Reasoning,3.933333333333333,73.33333333333333,
Causal_Consistency,4.7272727272727275,93.18181818181819,
Causal_Quality,4.366666666666666,84.16666666666666,
Spatial_Reasoning,3.66,66.5,
Spatial_Consistency,3.85,71.25,
Spatial_Quality,4.83,95.75,
Logical_Reasoning,1.423529411764706,10.588235294117649,
Logical_Consistency,3.070588235294118,51.76470588235295,
Temp-Life Progression,3.568421052631579,64.21052631578948,0.15789473684210525
Temp-Material Progression,3.473913043478261,61.84782608695652,0.21739130434782608
Temp-Environmental Cycles,3.9692307692307693,74.23076923076923,0.07692307692307693
Temp-Societal Transformation,4.0285714285714285,75.71428571428571,0.0
Causal-Structural Deformation,3.9833333333333334,74.58333333333333,0.25
Causal-State Transition,3.888,72.2,0.28
Causal-Chemical and Biological Transformation,4.15,78.75000000000001,0.25
Causal-Physics Manifestation,4.0,75.0,0.15384615384615385
Spa-Component Assembly,3.895652173913043,72.39130434782608,0.30434782608695654
Spa-Object Arrangement,3.6666666666666665,66.66666666666666,0.0
Spa-Viewpoint Generation,4.007407407407407,75.18518518518518,0.3333333333333333
Spa-Structural Inference,3.6266666666666665,65.66666666666666,0.2
Spa-Layout Reasoning,3.8434782608695657,71.08695652173914,0.30434782608695654
Logic-Pattern Prediction,1.203225806451613,5.080645161290326,0.03225806451612903
Logic-Mathematical Derivation,2.557142857142857,38.92857142857142,0.2857142857142857
Logic-Puzzle Solving,1.23,5.75,0.05

[ZpyWHU]

Hi, thank you very much for your detailed message and for your interest in RISEBench！

Before we proceed, we’d like to clarify two small points:

• Could you please let us know which model you used for your runs?
• In our paper’s Table.1, the reported metric is Accuracy(19.16), rather than Score-Origin or Score-Percentage.

[wyhlovecpp]

The model is Fluxkontext dev finetuned by our dataset. GPT-Image-Edit. I will test more results to make sure the evaluation is correct.