Research projects from Daphne Ippolito's lab at Carnegie Mellon University's Language Technologies Institute.

Last updated: July 2025

"Forcing Diffuse Distributions out of Language Models"
Yiming Zhang, Avi Schwarzschild, Nicholas Carlini, Zico Kolter, Daphne Ippolito

Addresses the problem that instruct LLMs exhibit strong biases when asked to generate random outputs, proposing a fine-tuing method to make models produce more uniform distributions for synthetic dataset generation.

"Chasing Random: Instruction Selection Strategies Fail to Generalize"
Harshita Diddee, Daphne Ippolito

Demonstrates that popular instruction selection strategies for LLMs generalize poorly across different datasets and evaluation setups, often failing to consistently outperform random baselines while incurring higher computational costs than training on full datasets.

"Human-Aligned Chess With a Bit of Search"
Yiming Zhang, Athul Paul Jacob, Vivian Lai, Daniel Fried, Daphne Ippolito

Introduces Allie, a chess AI that learns to ponder lke humans with a time-adaptive MCTS algorithm. Achieves state-of-the-art prediction of human behavior and strong skill adaptation against online players from 1000-2600 Elo.

"Persistent Pre-Training Poisoning of LLMs"
Yiming Zhang*, Javier Rando*, Ivan Evtimov, Jianfeng Chi, Eric Michael Smith, Nicholas Carlini, Florian Tramèr, Daphne Ippolito

Shows that 0.1% poisoning of LLM's pre-training data persists after alignment (SFT and DPO), and an attacker can achieve various goals such as denial-of-service, context extraction and belief manipulation with access to pre-training data alone.

"NoveltyBench: Evaluating Creativity and Diversity in Language Models"
Yiming Zhang, Harshita Diddee, Susan Holm, Hanchen Liu, Xinyue Liu, Vinay Samuel, Barry Wang, Daphne Ippolito

Benchmark for measuring how well language models generate multiple diverse, high-quality responses to queries involving subjectivity, randomness, and creativity. Evaluates 20+ frontier models including GPT-4o, Claude 3.5 Sonnet, and Gemini 2.0 Pro.

"CIE: Controlling Language Model Text Generations Using Continuous Signals"
Vinay Samuel, Harshita Diddee, Yiming Zhang, Daphne Ippolito

Continuous control signal approach for controlling attributes in LM outputs. Experiments ran for response word count attribute but designed to be extended to other attributes as well. We show stronger control of desired word count compared to previous approaches including prompting approaches.

"Command-V: Pasting LLM Behaviors via Activation Profiles"
Barry Wang, Avi Schwarzschild, Alexander Robey, Ali Payani, Charles Fleming, Mingjie Sun, Daphne Ippolito

"Finetune once, use on many LLMs" - the first method to demonstrate cross-architecture transfer of finetuned adapters without backpropagation. Works between Llama, Qwen, and other model families.