You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Post-training alignment often reduces LLM diversity, leading to a phenomenon known as <em>mode collapse</em>.
638
+
Unlike prior work that attributes this effect to algorithmic limitations, we identify a fundamental, pervasive data-level driver: <em>typicality bias</em> in preference data,
639
+
whereby annotators systematically favor familiar text as a result of well-established findings in cognitive psychology.
640
+
We formalize this bias theoretically, verify it empirically on preference datasets, and show that it plays a central role in mode collapse.
Motivated by this analysis, we introduce <strong>Verbalized Sampling (VS)</strong>, a simple, training-free prompting method to circumvent mode collapse. VS prompts the model to verbalize a probability distribution over a set of responses (e.g., "Generate 5 jokes about coffee and their corresponding probabilities").
644
+
Comprehensive experiments show that VS significantly improves performance across creative writing (poems, stories, jokes), dialogue simulation, open-ended QA, and synthetic data generation, without sacrificing factual accuracy and safety. For instance, in creative writing, VS increases diversity by 1.6-2.1× over direct prompting. We further observe an emergent trend that more capable models benefit more from VS.
645
+
In sum, our work provides a new data-centric perspective on mode collapse and a practical inference-time remedy that helps unlock pre-trained generative diversity.
646
+
</p>
647
+
</div>
648
+
</div>
649
+
</div>
650
+
</section>
651
+
615
652
{/* Verbalized Sampling: Title & Description left, install/code right */}
Post-training alignment often reduces LLM diversity, leading to a phenomenon known as <em>mode collapse</em>.
680
-
Unlike prior work that attributes this effect to algorithmic limitations, we identify a fundamental, pervasive data-level driver: <em>typicality bias</em> in preference data,
681
-
whereby annotators systematically favor familiar text as a result of well-established findings in cognitive psychology.
682
-
We formalize this bias theoretically, verify it empirically on preference datasets, and show that it plays a central role in mode collapse.
Motivated by this analysis, we introduce <strong>Verbalized Sampling (VS)</strong>, a simple, training-free prompting method to circumvent mode collapse. VS prompts the model to verbalize a probability distribution over a set of responses (e.g., "Generate 5 jokes about coffee and their corresponding probabilities").
686
-
Comprehensive experiments show that VS significantly improves performance across creative writing (poems, stories, jokes), dialogue simulation, open-ended QA, and synthetic data generation, without sacrificing factual accuracy and safety. For instance, in creative writing, VS increases diversity by 1.6-2.1× over direct prompting. We further observe an emergent trend that more capable models benefit more from VS.
687
-
In sum, our work provides a new data-centric perspective on mode collapse and a practical inference-time remedy that helps unlock pre-trained generative diversity.
@@ -785,24 +783,24 @@ export default function HomePage() {
785
783
<strong>Figure 4:</strong> Qualitative and quantitative examples of Verbalized Sampling on creative writing, dialogue simulation, and enumerative open-ended QA.
Our comprehensive experiments on multiple tasks demonstrate that Verbalized Sampling significantly improves the diversity-quality trade-off across tasks and model families,
791
-
without compromising factual accuracy and safety.
792
-
</p>
793
-
<p>
794
-
As shown in Figure 4, for <strong>story writing</strong>, VS improves the output diversity.
795
-
For <strong>dialogue simulation</strong>, VS simulates the donation amount distribution much closer to the human distribution, and generates more realistic persuasion behaviors.
796
-
On the task of <strong>enumerative open-ended QA</strong>, we ask the model to "generate US states". We first query a pretraining corpus (RedPajama) to establish a "reference" distribution of US
797
-
state names in the pretraining data. The verbalized probability distribution generated by VS, when averaged over 10 trials, closely aligns with this reference pretraining distribution (KL=0.12).
798
-
In contrast, direct prompting collapses into a few modes, repeatedly outputting states like California and Texas.
Our comprehensive experiments on multiple tasks demonstrate that Verbalized Sampling significantly improves the diversity-quality trade-off across tasks and model families,
789
+
without compromising factual accuracy and safety.
790
+
</p>
791
+
<p>
792
+
As shown in Figure 4, for <strong>story writing</strong>, VS improves the output diversity.
793
+
For <strong>dialogue simulation</strong>, VS simulates the donation amount distribution much closer to the human distribution, and generates more realistic persuasion behaviors.
794
+
On the task of <strong>enumerative open-ended QA</strong>, we ask the model to "generate US states". We first query a pretraining corpus (RedPajama) to establish a "reference" distribution of US
795
+
state names in the pretraining data. The verbalized probability distribution generated by VS, when averaged over 10 trials, closely aligns with this reference pretraining distribution (KL=0.12).
796
+
In contrast, direct prompting collapses into a few modes, repeatedly outputting states like California and Texas.
0 commit comments