Week 6 Memo

[avioberoi]

Post a memo that explores the readings and class topics relevant to your final project by Thursday, Feb 12, 11:59 PM. The memo should be 300–500 words and include the following: (1) state your research question (for the imagined final project) succinctly in a single sentence at the beginning (this can and should evolve across the weeks of the quarter as the project becomes more concrete); (2) propose a research design that helps to address that question, and not proposed in any prior memo, informed by at least one of the required readings and one of the supplemental readings from this week; (3) a visual figure or diagram that draws upon pilot (nonhallucinated) data, simulated data with defensible assumptions, or provides a clear conceptual illustration that persuades the reader (aka James and TAs) of the appropriateness and fruitfulness of the design to address your question. You will then pilot this design as the final question in the Week 2 Homework due the following Wednesday.

By 10AM Friday, each student will up-vote (“thumbs up”) what they think are the five most interesting memos for the week!

Required:
AI Foundations: “Towards Monosemanticity: Decomposing Language Models With Dictionary Learning.” Trenton Bricken, Adly Templeton, Joshua Batson,… Shan Carter, Tom Henighan, Chris Olah. 2023.
AI Designs: “Sparse Autoencoders Find Highly Interpretable Features in Language Models.” Hoagy
Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, Lee Sharkey. 2023.
Social Designs: “Linear representations of political perspective emerge in large language models”. Kim, J., Evans, J., & Schein, A. ICLR. 2025.
“Persona vectors: monitoring and controlling character traits in language models”. Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. arXiv. 2025.

Supplemental
AI Foundations: “Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet.” Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., ... & Olah, C. Anthropic. 2024.
“Mapping the Mind of a Large Language Model.” Anthropic. 2024. [Accessible companion to the technical work]
“A Primer on the Inner Workings of Transformer-based Language Models.” Ferrando, J., Sarti, G., Bisazza, A., & Costa-jussà, M. R. arXiv. 2024.
AI Designs: “Activation Addition: Steering Language Models Without Optimization.” Turner, A., Thiergart, L., Udell, D., Leech, G., Mini, U., & MacDiarmid, M. arXiv. 2023.
“Inference-Time Intervention: Eliciting Truthful Answers from a Language Model.” Li, K., Patel, O., Viégas, F., Pfister, H., & Wattenberg, M. NeurIPS. 2023.
“In-Context Vectors: Making In-Context Learning More Effective and Controllable Through Latent Space Steering.” Liu, S., Lei, C., Huang, X., & Huang, Y. ICML. 2024.
“Scaling and Evaluating Sparse Autoencoders.” Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., ... & Wu, J. OpenAI. 2024.
“Transcoders Find Interpretable LLM Feature Circuits.” Dunefsky, J., Chlenski, P., & Nanda, N. arXiv. 2024.
Social Designs: “Uncovering Latent Arguments in Social Media Messaging by Employing LLMs-in-the-Loop Strategy.” Fang, H., Lim, A., Chew, R., & Bian, J. arXiv. 2024.
“Eliciting Human Preferences with LLMs-as-Questionnaires.” Reddy, S., Michael, J., Ziegler, D., et al. arXiv. 2024.

[leahdimsu]

Week 6 Memo: Feature Activation Patterns Reveal Review Authenticity

Research Question: Do sparse autoencoder (SAE) feature activations trained on Amazon reviews reveal interpretable "authenticity vectors" that distinguish genuine product experiences from AI-generated or incentivized content, and can these vectors predict whether reviews were written pre- vs. post-2023 (the ChatGPT inflection point)?

Research Design

Recent work on monosemanticity demonstrates that SAEs can decompose LLM activations into interpretable features (Bricken et al., 2023; Templeton et al., 2024). I extend this to a social science context: using SAEs to discover latent dimensions of review authenticity.

Method: Train a sparse autoencoder (128-512 features, L1 penalty α=0.01) on BERT embeddings of Amazon video game reviews (n≈100,000). Following Cunningham et al. (2023), I identify highly interpretable features by: (1) computing feature activation patterns across reviews; (2) extracting top-activating examples for each feature; (3) manually annotating features as "authenticity markers" (specific gameplay details, emotional authenticity, temporal references) vs. "generic patterns" (boilerplate praise, feature lists).

Key Innovation: Apply "activation addition" steering (Turner et al., 2023) in reverse rather than injecting authenticity vectors to modify model outputs, I extract authenticity vectors from human-written reviews to detect AI content. Reviews are projected onto discovered feature vectors; those with low activation on experiential features (e.g., "specific bug descriptions," "playtime mentions") but high activation on generic features (e.g., "professional product review language") are flagged as potentially synthetic.

Temporal Test: Reviews pre-2023 should show stable feature activation patterns. Post-2023 reviews should show: (1) decreased activation on experiential features; (2) increased activation on "coherent but generic" features; (3) emergence of new SAE features corresponding to AI writing patterns (e.g., excessive hedging, unnatural transitions). This mirrors Kim et al.'s (2025) finding that political perspectives emerge as linear representations, here, temporal authenticity emerges as interpretable SAE features.

Connection to Readings: Bricken et al. (2023) show SAEs find monosemantic features in LLMs; I show they find monosemantic features in human text corpora. Cunningham et al. (2023) validate interpretability via human annotation; I validate via temporal prediction (pre/post-ChatGPT split). Reddy et al.'s (2024) "LLMs-as-Questionnaires" suggests LLMs can elicit preferences; my design suggests SAEs-as-Detectors can elicit authenticity signals humans miss.

Visual Element

The accompanying three-panel visualization demonstrates: (A) Feature Discovery: Heatmap showing top 20 SAE features ranked by interpretability score (activation entropy × human annotation agreement). Features cluster into "Experiential" (gameplay mechanics, bugs, playtime), "Emotional" (frustration, joy, nostalgia), and "Generic" (product features, comparisons). (B) Temporal Shift: Line plot (2018-2024) showing declining activation on experiential features post-2023, with 95% bootstrap CIs. The 2023 inflection point is stark with experiential feature activation drops 38% while generic feature activation rises 52%. (C) Authenticity Space: 2D t-SNE projection of reviews onto top-2 authenticity eigenvectors (derived from SAE features). Pre-2023 reviews (blue) cluster densely around high experiential activation; post-2023 reviews (red) show bimodal distribution—one cluster overlaps authentic reviews (genuine human), another cluster distinct region (likely AI). Convex hull boundaries suggest ~15-20% of post-2023 reviews occupy "AI writing" region.

Why This Matters: Current AI detection tools use surface statistics (perplexity, burstiness). This design leverages interpretable latent semantics which is what reviewers talk about, not how they write. SAE features provide mechanistic explanations (e.g., "Review flagged because Feature #47 'specific bug descriptions' activated weakly") rather than black-box scores. This addresses Anthropic's call for interpretability-first AI safety (Templeton et al., 2024) by showing interpretability methods solve real social science problems: detecting synthetic content at scale.

If successful, this design yields: (1) deployable authenticity detection for platforms; (2) evidence that ChatGPT fundamentally altered review ecosystems; (3) proof that SAE interpretability methods generalize beyond LLM internals to human text analysis. The approach is novel as no prior work applies monosemanticity to corpus-level authenticity detection and immediately implementable given existing Amazon review data and open-source SAE implementations.

[yuanzeliu]

Research Question.

Does fact-based reasoning (FBR) training causally reorganize the latent reasoning-style features of model-generated political text, and can sparse autoencoders (SAEs) recover interpretable dimensions that reflect this restructuring?

Background

Across prior weeks, I have been developing a large-scale simulation of partisan agents interacting over controversial political statements. The broader goal of the final project is to understand whether FBR-style interventions reduce polarization by altering how agents reason, rather than merely shifting surface agreement. Recent advances in mechanistic interpretability suggest that language models contain sparse, semantically coherent internal features that can be extracted through dictionary learning (Bricken et al., 2023; Cunningham et al., 2023). Complementing this, Kim, Evans, and Schein (2025) show that political perspective emerges linearly in representation space, and Chen et al. (2025) demonstrate that persona traits can be monitored and steered via latent vectors. Together, these works motivate examining whether FBR induces a measurable reconfiguration in latent reasoning space.

Research Design

This week’s design builds directly on the simulation dataset constructed earlier in the quarter. In that task, partisan agents (Democrat or Republican personas) evaluated and reasoned about a diverse set of political statements across multiple rounds. The dataset contains structured metadata (subject_id, persona, condition, topic, congruence, stage) and free-text reasoning outputs (thought_process). The key manipulation occurs across stages: a baseline phase and a transfer phase following FBR exposure.

To isolate reasoning-style shifts rather than topic differences, I first restrict analysis to baseline and transfer outputs and balance them within topic. Sentence-level embeddings are computed for each reasoning text. To further remove topic content variance, embeddings are centered within topic before feature extraction. I then train a sparse autoencoder on these residualized embeddings to recover latent features, following the dictionary-learning framework of Bricken et al. (2023). Features are selected by stage difference (transfer minus baseline), and GPT-4o is used in a contrastive labeling setup—receiving high- versus low-activation exemplars—to assign interpretable names describing what increases with activation. Finally, I estimate robust OLS models to quantify transfer effects on each latent feature.

Visual Evidence

The accompanying figure plots transfer-minus-baseline effects on SAE-derived reasoning features, estimated with HC3 robust standard errors. The results indicate that FBR exposure is associated with increases in dimensions labeled as Nuanced Acknowledgment, Evidence-Based Alignment, Conditional Reasoning, and Policy Debate Engagement. At the same time, features labeled Ideological Framing, Empirical Reasoning, Nuanced Causal Analysis, and Partisan Alignment Reasoning show higher activation during baseline than transfer. Importantly, these labels do not refer to policy topics but to abstract reasoning styles extracted from topic-balanced embeddings. The pattern suggests that FBR does not simply increase factual references; instead, it appears to reorganize the structure of reasoning, shifting from loosely ideological or empirically descriptive framing toward more explicit, structured, and debate-oriented forms of argumentation. This supports the hypothesis that interventions may operate in latent reasoning space by redistributing activation across sparse semantic features rather than along a single “truthfulness” axis.

References

Bricken, T., Templeton, A., Batson, J., et al. (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning.

Cunningham, H., Ewart, A., Riggs, L., Huben, R., & Sharkey, L. (2023). Sparse Autoencoders Find Highly Interpretable Features in Language Models.

Kim, J., Evans, J., & Schein, A. (2025). Linear representations of political perspective emerge in large language models. ICLR.

Templeton, A., et al. (2024). Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet.

Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. (2025). Persona vectors: monitoring and controlling character traits in language models.

[yangyuwang]

Research Question

Can I interpret and steer the neurons in caption-to-painting Latent Diffusion Models (LDMs) based on contextual metadata (artist gender, creation location, and year), and do linear representations of these attributes exist within the model's U-Net architecture?

Research Design

Building on recent interpretability research in Large Language Models (LLMs)—specifically persona vectors and activation addition—I propose to investigate whether similar steering mechanisms can be applied to computer vision models. This design tests the hypothesis that diffusion models encode semantic meaning (such as gender or historical context) in a way that can be isolated and manipulated via linear steering in activation space.

Data Construction and Processing. The foundation of this study is a custom-curated dataset of 7,840 digitized artworks sourced from WikiArt, filtered to capture the distinct stylistic transition period between 1400 and 1600. To enable precise steering, I constructed a multi-modal annotation pipeline that pairs raw pixel data with both descriptive content and high-level contextual metadata.

I utilized a dual-LLM approach to generate these annotations. First, GPT-4o was employed to process unstructured biographical text from artists' Wikipedia pages. This pipeline extracted and standardized three primary steering variables: Artist Gender (binary: Male/Female), Creation Location (geographic contrast: China vs. Italy), and Creation Year (temporal contrast: centered around 1420 vs. 1620). This automated extraction was crucial for converting noisy historical records into clean, steerable attributes.

Second, to ensure the model learns to disentangle style from subject matter, I generated dense visual captions for every image using LLaVA (Large Language-and-Vision Assistant). These captions provide the model with explicit text descriptions of the visual content (e.g., "a portrait of a woman in a red dress"), allowing us to isolate the specific influence of the metadata tags during the steering experiments. All images were pre-processed and resized to a resolution of 128x128 pixels to facilitate efficient training of the Denoising U-Net from scratch.

Interpretability Strategy. To validate whether the model's internal representations can be steered, I employed three distinct analytical methods:

1. Activation Patching: Inspired by causal tracing in LLMs (Meng et al., 2022), this method aims to automatically identify specific U-Net layers that causally control the generation of specific attributes.

2. Neuron Discovery: Drawing on Sparse Auto-encoder research (Cunningham et al., 2023; Bricken et al., 2023), this approach seeks to locate specific individual neurons (weights) that encode attributes like gender or location by analyzing weight differences in the output layers.

3. Linear Steering: Inspired by research on linear representations of political perspectives (Kim et al., 2025), this method attempts to force the model in a certain contextual direction (e.g., -2.0 Female to +2.0 Male) by applying activation-difference vectors to the middle blocks of the U-Net.

Figures

Fig. 1. Steering Vector (Δactivation) Seaching Framework in LDM U-Nets

Fig. 2. Linear Steering Results based on the Steering Vector (output = output + α * steering_vector)

[YanjingAveryLi]

Do painting styles evolve along one main trajectory, or through multiple branching and overlapping trajectories in embedding space?

Analysis Framework and Design

Art history is often narrated as a tidy sequence—Renaissance to Baroque to Romanticism to Impressionism to Cubism—which implies that style progresses along a single main path. This memo tests an alternative account: that each style occupies a region in embedding space with internal geometry, including a dense core of canonical works, lower-density arms associated with substyles or emphasized formal cues, and bridge zones where neighboring styles overlap. Recent work showing that high-dimensional learned representations can sometimes encode relatively linear global structure, but can also contain richer, locally interpretable organization that is not reducible to a single axis (Bricken et al., 2023; Cunningham et al., 2023; Kim et al., 2025). Therefore, we propose if stylistic change were truly linear, one dominant global direction should explain most transitional works and cross-style influence. If style instead has richer topology, change should appear as local branching, overlap, and recombination.

To study this question, I use digitized paintings with style metadata from WikiArt. Each painting is embedded with two pretrained vision encoders, and all vectors are L2-normalized. As an initial diagnostic, I visualize the embeddings with a 3D PCA projection, coloring each point by style label. This projection is not the main test, but it provides a useful first check: if styles were organized along a single trajectory, one would expect a relatively ordered progression across the cloud. By contrast, visible overlap would suggest that style change is not well described by one global direction. Framed this way, the design parallels recent efforts to study whether conceptually meaningful differences in representation space are well captured by a dominant linear direction or instead require more structured geometric descriptions (Kim et al., 2025).

The main evidence comes from local neighborhood structure in the full embedding space. I run $k$-nearest-neighbor search (for example, $k=20$) and retain only mutual kNN pairs, producing a graph in which nodes are paintings and edges represent stable local similarity relations. Edge weights are based on cosine similarity. I then identify each movement’s core using local density measures such as $k$-core decomposition, and extract arms as sparse, chain-like extensions from the dense center using minimum spanning tree or skeletonization procedures. This yields a directly testable representation of whether styles are compact, branching, or overlapping.

To operationalize overlap, I define bridge pairs as stable edges connecting paintings with different style labels. For each painting $i$, I compute a bridge score:

$$ \mathrm{Bridge}(i) = \frac{{j \in NN(i): \mathrm{style}(j) \neq \mathrm{style}(i)}}{k}. $$

High-bridge regions are plausible transition zones: works that remain locally legible relative to their own style while also connecting to neighboring styles. This creates a clear comparison between two geometric accounts of stylistic change. Under the single-trajectory hypothesis, transitional works should align with one dominant axis, and most apparent overlap should be reducible to progression along that shared direction. Under the branching-topology hypothesis, by contrast, transitional works should cluster in local overlap neighborhoods, bridge specific arms or cores, and resist reduction to a single global ordering.

To ensure that distance reflects style rather than subject matter, I stratify analyses by genre (for example, portrait versus landscape) and require cross-encoder agreement as a robustness check. The goal is therefore not only to visualize style space, but to determine whether its structure is better explained by one main trajectory or by multiple branching and overlapping pathways.

Expected Patterns and Interpretation

The first figure, a 3D PCA projection, serves as a diagnostic. If style evolution followed one main path, styles should appear more cleanly ordered along a shared direction. If substantial overlap remains even in this simplified view, that already weakens the single-trajectory account.

The second figure, a 2D point-cloud projection, highlights representative bridge points alongside several movements. If these bridge points concentrate in local overlap regions rather than along one global gradient, that supports the branching account.

The third figure shows the mutual kNN graph with stable local edges. Its purpose is to test whether stylistic structure is organized by local neighborhoods rather than by one linear ordering. If styles evolve through multiple pathways, the graph should reveal many stable local relations that do not collapse into a single sequence.

The fourth figure provides the strongest test by visualizing cores, arms, and bridge zones. If painting styles evolve through multiple branching and overlapping trajectories, movements should exhibit dense canonical cores, uneven peripheral arms, and cross-label bridge edges concentrated in transition zones. Together, these patterns would support a topological account of stylistic change: artistic influence would be better understood as local convergence and recombination in embedding space than as movement along one historical line.

Figure

Figure 1. 3D PCA projection of painting embeddings (PC1–PC3). Each point is a painting, and colors indicate coarse style labels. Visible overlap suggests that styles are not separated by a single global direction.

Figure 2. 2D projection of painting embeddings. Points are paintings and colors indicate movement labels. Representative bridge points highlight local overlap regions between styles.

Figure 3. Stable nearest-neighbor graph. Edges are mutual-kNN pairs retained under stability filtering, making local structure directly testable.

Figure 4. Core/arm decomposition and bridge zones. Dense subgraphs define movement cores, sparse extensions define arms, and cross-label stable edges identify overlap and transition zones.

Reference

Bricken, T., Templeton, A., Batson, J., Carter, S., Henighan, T., & Olah, C. (2023). Towards monosemanticity: Decomposing language models with dictionary learning. arXiv preprint.
Cunningham, H., Ewart, A., Riggs, L., Huben, R., & Sharkey, L. (2023). Sparse autoencoders find highly interpretable features in language models. arXiv preprint.
Kim, J., Evans, J., & Schein, A. (2025). Linear representations of political perspective emerge in large language models. International Conference on Learning Representations (ICLR).

[michellecdu]

Research Question

How does scaling internal political vectors influence model decision-making? Specifically, can we scale a political “in-context vector” (ICV) to create varying “ideological intensities,” and does the model’s resource allocation shift linearly based on the intensity of steering? This study aims to investigate whether political extremism in LLMs, modeled as a simple magnitude shift in latent space, correlates with social friction in real-world fiscal policymaking scenarios.

Research Design
This experiment investigates whether “liberal” or “conservative” perspectives are measurable, linear directions in a model’s internal geometry. It extends upon previous research that has found that linear probes can steer model outputs toward certain political ideologies, suggesting that LLMs possess high-level linear representations of “conservative” or “liberal” stances (Kim et al., 2025).

First, I will isolate a political ICV by capturing the contrastive latent shift in Gemma-2-2b (v_pol = act_cons - act_lib). I will record the model’s internal activations across two opposing sets of partisan statements and calculate the mean difference vector at Layer 18 (Liu et al., 2024). Then, I will use Activation Addition (ActAdd) to inject this “conservative” – “liberal” steering vector during inference (Turner et al., 2024). By varying a scaling coefficient alpha, I will create five agent profiles: two “hyperpartisan” versions (extreme left/right), two “moderate” versions, and one unmodified “centroid” (neutral).

The experiment consists of two parts. First, each agent variant will perform allocation tasks, distributing funds between competing priorities (e.g., social services vs. law enforcement) across hundreds of randomized scenarios. I will track the rate of behavioral change relative to the strength of the scaling coefficient alpha in each direction (“steering sensitivity”) and the specific dollar-value shift in resource distribution between ideologically-opposed resource allocation categories.

Second, I will pair these agents in multi-turn negotiation simulations where two agents with distinct ideological profiles attempt to reach a consensus on a shared budget. By pairing agents with varying steering intensities, I can calculate the “negotiation gap,” or the mean difference between the agents’ final budget proposals after a fixed number of turns. This metric allows me to correlate latent vector distance with the degree of fiscal disagreement.

This research moves beyond qualitative descriptions toward a mechanistic understanding of how internal model geometry dictates the outcomes of dynamic social and economic interactions.

Pilot Figures
I assessed the ICV by prompting steered agents to perform a budget allocation task. By asking the agents to specify a million-dollar expenditure (0–10) for Social Welfare, I mapped the "dose-response" relationship between internal vector magnitude and external policy preference (social welfare). Figure 1 shows exactly how much the model’s behavior shifts, on average, per unit of internal steering. It illustrates that activation steering has a relatively steady impact on fiscal allocation across the ideological spectrum, but extreme steering intensities may introduce behavioral fluctuations. Figure 2 uses PCA to capture the internal activations of agent profiles based on the steering intensity alpha. We can see that the steering vector creates a linear trajectory from liberal to conservative poles within latent space, confirming Kim et al.

Figure 1. Dose-Response Curve: Fiscal Allocation Decision by Steering Alpha

Figure 2. Principal Component Analysis of Latent Trajectories Under Political Activation Steering

References
Liu et al. (2024). “In-Context Vectors: Making In-Context Learning More Effective and Controllable Through Latent Space Steering.” ICML'24: Proceedings of the 41st International Conference on Machine Learning. https://dl.acm.org/doi/10.5555/3692070.3693379
Kim et al. (2025). “Linear representations of political perspective emerge in large language models.” International Conference on Learning Representations (ICLR) 2025. https://proceedings.iclr.cc/paper_files/paper/2025/file/1557cc3e1a67f985dbdb162db9963dd0-Paper-Conference.pdf
Turner et al. (2024). “Activation Addition: Steering Language Models Without Optimization.” arXiv. https://arxiv.org/html/2308.10248v4

[shawnkuolo]

Week 6 Memo

Research Question

Do RAG-augmented LLMs develop linear representations of disciplinary epistemology in their activation space, and can sparse autoencoder features identify the specific internal mechanisms through which domain knowledge (CS vs. Sociology) shapes the causal reasoning style of generated questions?

Research Design: Linear Probing for Disciplinary Epistemology

My project uses CHORUS’s disciplinary RAG pipeline (CS, Sociology, Hybrid) to generate causal inference questions and examines whether the LLM’s internal representations encode disciplinary epistemology as a recoverable linear direction. This week’s interpretability readings motivate a different approach than Week 5’s active learning classifier: rather than classifying outputs, I propose looking inside the model to understand how disciplinary knowledge reshapes LLM cognition.
Kim, Evans, and Schein (ICLR 2025) demonstrate that LLMs possess linear representations of political perspective—probes trained on lawmaker ideology generalize to predict news outlet slant. I adapt this to a disciplinary axis: where they probe for liberal–conservative, I probe for CS–Sociology epistemological orientation. I will (1) prompt Llama-3.1-8B with CHORUS-CS and CHORUS-SOC RAG contexts to generate 200 causal questions each, (2) extract residual stream activations at every layer, and (3) train logistic regression probes to classify disciplinary origin. Following Kim et al.’s finding that middle layers encode high-level concepts, I expect peak separability around layers 14–18.
The Anthropic monosemantic features work (Bricken et al. 2023; Templeton et al. 2023, required reading) provides the complementary tool: sparse autoencoders (SAEs) decompose polysemantic activations into interpretable features. After identifying where disciplinary probes perform best, I will train an SAE on those layers and identify which dictionary features differentially activate for CS vs. SOC contexts. This moves beyond classification to mechanistic explanation: which internal “concepts” does the model activate when reasoning with sociological versus computational causal frameworks? Bricken et al. show SAE features are more interpretable than raw neurons, which is critical for characterizing disciplinary epistemology at a feature level.
The persona vectors framework (Lindsey et al. 2025, supplemental) further suggests that behavioral traits can be extracted as activation-space directions for monitoring and steering. By analogy, a “disciplinary epistemology vector” from CHORUS could enable steering toward more sociological or computational reasoning—a direct intervention impossible with output-only classification.
This design is immediately actionable. The pilot (Figure 1C): (1) sample 50 CS and 50 SOC RAG chunks from CHORUS’s FAISS index; (2) prompt Llama-3.1-8B to generate one causal question per chunk; (3) extract residual stream activations at all 32 layers via TransformerLens; (4) train per-layer logistic regression probes; (5) PCA-visualize the peak-accuracy layer. If probes achieve above-chance accuracy (as simulated in Figure 1B, based on the Kim et al. pattern), this validates that disciplinary RAG knowledge leaves a linearly detectable trace—justifying full SAE decomposition in subsequent weeks.

Figure 1. (A) Simulated PCA of LLM activations when processing CS vs. SOC RAG contexts, showing expected linear separability along a disciplinary direction. (B) Simulated per-layer probe accuracy, with middle layers predicted to be most discriminative (following Kim et al. 2025). (C) Pilot experiment pipeline for the Week 6 coding presentation.

Haixiao Luo

[jhixon2]

Research Question:

Do Twitch chat communities form geometrically structured representations in BERT’s activation space, and does this structure align with linguistic or audience-based interpretations of community?

Research Design:

Linear probes trained on the activations of an un-finetuned BERT model were tasked with classifying samples of 20 chat messages to one of 27 streamers. The linear probing revealed that accuracy peaked in layer 4 at 0.636, and that the un-finetuned BERT model best captures the distinctions between different streamers’ Twitch chats in its middle layers (Fig. 1). This result is consistent with Kim et al.’s results which found that middle layers were the most predictive when probing LLMs promoted to generate text from lawmakers with different political ideologies (2025).

Following this, the BERT model was fine-tuned on a training set of Twitch chat messages. With the fine-tuned model, predictive accuracy increased across layers. Accuracy peaked in layer 12 at 0.931, and attention heads became uniformly accurate after layer 8 (Fig. 3). Following Li et al.’s approach of projecting activation spaces into 2 dimensions to visualize geometric structures (2023), I applied a UMAP to layer 12’s activation space, which depicted the formation of distinct geometric clusters for each streamer (Fig. 4). The geometric clustering of streamers in the fine-tuned BERT classifier’s activation space was missing in the UMAP of the un-finetuned BERT model (Fig. 2), demonstrating that through training, the model was able to learn geometrically structured representations of Twitch chats.

The streamers in fine-tuned BERT’s layer 12 activation space were then colored based on their assigned clusters determined by word-embeddings (Fig. 5). The gaming streamers (blue) were more tight-knit, but with some variance, whereas the variety streamers (green) were more spread out. This makes sense since variety streamers, as the name suggests, stream a variety of different content, while gaming streamers all mostly stream competitive games, but vary in the games that they play, with some games being more popular than others.

When the streamers were colored based on their clusters determined by audience similarity, there was much less visible geometric structure (Fig. 6). This suggests that the fine-tuned BERT’s activation space reflects linguistic rather than audience-based community structure, which implies that shared language and shared chatters represent two distinct dimensions of Twitch communities.

The finding that streamer classification based on chat messages aligns better with linguistic features than audience motivates the question of to what extent do chatters change their language and vocabulary when participating in different streamers’ chats? Do they bring parts of other communities or their home communities with them, or do they completely assimilate into each streamer’s chat culture? Does a chatter who frequents streams from streamers with distinctly different chat cultures stand out from the other chatters, or do they change the way that they chat? Is the behavior different for each chatter, or do the majority of all ‘wandering’ chatters stand out or assimilate? These are important questions to investigate to later be able to design chatter agents that behave like real humans.

Figure:

[baihuiw]

Research question: Do large language models exhibit cognitive dissonance in moral judgment and is this cognitive dissonance linearly encoded in intermediate hidden-state representations?

Prior memos examined confirmation bias as a scalar quantity — how much does the model shift. This memo proposes a distributional mechanism design: rather than measuring mean Δ rating, I characterize where probability mass flows across the full 1–7 response distribution, conditional on the configuration of cue and prior. This reframing is motivated by a key observation in pilot data: two conditions that produce near-identical mean shifts can reflect entirely different internal computations.

The design compares three cue–prior configurations derived from Kim, Evans & Schein (2025)'s framework of directional political representations. If LLM judgments have a geometric structure in representation space, then cue responses should not be scalar but directional — and the direction of redistribution should differ systematically by condition. Concretely: (1) PULL — neutral prior, cue pushes toward a side; (2) OPPOSE — held prior, cue pushes against it; (3) REINFORCE — held prior, cue pushes further in the same direction. The key testable prediction, informed by Cunningham et al. (2023)'s finding that sparse feature directions in transformer residual streams correspond to stereotyped behavioral tendencies, is that OPPOSE does not produce persuasion — it produces a locally constrained redistribution that is geometrically distinct from PULL and identifiable as a separate feature direction.

The pilot data below tests this prediction directly on GPT-4o responses to 78 moral dilemmas across 464 cue trials.

PULL: probability mass moves cleanly toward cue target (model just follows)
OPPOSE LEFT (prior=5-7, cue→left): model doesn't move toward cue at all — instead drops probability at position 7 and redistributes to adjacent positions 3-6. It softens its extreme without accepting the cue
OPPOSE RIGHT (prior=1-3, cue→right): similarly, drops position 1, redistributes to positions 2-4 — again compromise without capitulation
REINFORCE: small movement further toward the already-held direction

The procedure for next stage: (1) collect hidden states at 7 sampled layers (−30 through −1) for each of 546 Study 1 requests across memory-only and memory+cue conditions; (2) for each layer, train a ridge-regularized linear probe predicting the signed response shift (Δ = answer − prior); (3) compare probe accuracy across layers to identify where bias information is maximally encoded. Following Bricken et al. (2023)'s insight that mid-to-late residual stream features carry the richest semantic content, I expect probe accuracy to peak at layers −5 to −10, before the final logit projection at layer −1 collapses the representation.

Crucially, this design tests a specific mechanistic claim: if the probe at layer −10 predicts the final shift with above-chance accuracy, this is evidence that cognitive dissonance is not a surface-level decoding artifact but a stable internal computation — making it a candidate target for activation steering interventions.

Kim, Evans & Schein (2025) demonstrate that political perspective is linearly decodable from intermediate LLM layers, and that this representation is causally upstream of behavior. This grounds the assumption that moral judgment — like political perspective — may have a stable linear structure in the residual stream, detectable before output generation.

Bricken et al. (2023) show that features in mid-to-late transformer layers are more semantically interpretable than early layers, and that linear decomposition of these features reveals human-interpretable concepts. This motivates the expected inverted-U shape in panel C: early layers have insufficient semantic richness; the final layer collapses representations into logit space. The peak is expected around layers −10 to −5.

Taken together, the pilot data in panels A and B establish that (a) Llama-3-8B has a strong neutral baseline (mode = 4, 55% of items) that makes cue-induced shifts detectable against a stable prior, and (b) GPT-4o exhibits the asymmetric resistance pattern (0.38× reinforce vs. oppose shift) that a representational probe should capture internally. The proposed design tests whether this asymmetry is geometrically encoded — and at which layer.

[mdsowder]

Research Question:

What narrative dimensions underlie neighborhood crime perception, and how do these dimensions relate to spatial disparities between perceived and actual crime?

My thesis examines the spatial gap between perceived and actual crime across Chicago neighborhoods. This week’s readings on monosemanticity, sparse autoencoders, and linear political representations suggest a new research design: instead of treating crime perception as a scalar survey outcome, I propose decomposing it into interpretable semantic features extracted from language.

Drawing on Bricken et al. (2023) and Cunningham et al. (2023), I propose using a sparse autoencoder (SAE) applied to an embedding space of neighborhood-level perception texts (e.g., open-ended CHA responses, NORC sentiment responses if accessible, or simulated text reflecting neighborhood discourse). The goal is to learn sparse, monosemantic features—dimensions that correspond to interpretable concepts such as disorder, police presence, youth activity, or racialized threat framing. This moves beyond black-box embeddings toward interpretable feature discovery.

Inspired by Kim et al. (2025), who show that political perspective emerges linearly in representation space, I will test whether a “perceived danger” vector similarly emerges linearly across neighborhood texts. If so, I can construct a perception-direction vector and project each neighborhood’s discourse onto it. This allows for a spatialized semantic index of perceived crime, independent of actual crime rates.

Finally, informed by Chen et al. (2025) on persona vectors, I will test whether manipulating latent features (e.g., amplifying “disorder” or “collective efficacy” features) systematically shifts predicted perception scores. This provides a causal interpretability check: do these learned features meaningfully drive perceived crime?

Empirically, I will then:

1. Extract neighborhood-level text embeddings.
2. Train a sparse autoencoder to identify interpretable features.
3. Construct linear perception vectors.
4. Compare semantic projection scores to actual CPD crime rates using spatial regression (e.g., spatial lag/error models).

The key test: do certain semantic features spatially cluster in ways that diverge from actual crime patterns? If so, this suggests that perception gaps are structured by interpretable narrative dimensions rather than crime incidence alone.

Visualizations

Figure 1: Shows that while perceived violence is positively correlated with objective crime rates (z-standardized), substantial dispersion exists across neighborhoods. Several community areas (e.g., Hyde Park, Brighton Park) exhibit elevated perceived violence relative to their crime rates, while others (e.g., Loop) display comparatively lower perception than expected.

[JonathanPMonroe]

Research Question

Can AI models explain why social ties form by representing edge-level decisions through a small number of interpretable social mechanisms?

Research Design and Connection to Readings

Many models used for social network analysis, such as link prediction or edge imputation, are optimized for accuracy but make it difficult to understand why a specific social tie is predicted or not. This project shifts attention away from aggregate performance and instead treats each potential tie as a decision that should be explainable.

Recent work on AI interpretability suggests that complex models often rely on a small number of meaningful internal features. Bricken et al. (2023) and Cunningham et al. (2023) show that sparse autoencoders can recover features in language models that correspond to single, interpretable concepts rather than dense, entangled representations. Templeton et al. (2024) further show that these interpretable features remain stable and meaningful even as models scale, suggesting that model behavior can often be understood through a limited set of consistently activated signals. In a social context, Kim, Evans, and Schein (2025) demonstrate that political perspectives in language models often lie along simple linear directions, while Chen et al. (2025) show that behavior can be summarized and controlled using compact “persona vectors.” Together, these papers suggest that model behavior can often be explained using a few weighted features rather than many opaque ones.

To examine this idea in a social context, I use the Email-EuAll directed email network, which records who emailed whom within a large research institution. This dataset is well-suited to edge-level explanation because email ties are directional, sparse, and decision-like; each edge reflects an individual choice to initiate communication. I sample individual candidate edges u → v, including both observed ties and missing ties. For each candidate, I compute a small set of intuitive structural signals: the number of shared contacts between u and v (triadic closure), sender activity, receiver popularity, reciprocity, and neighborhood overlap.

Rather than fitting a single black-box model and reporting global accuracy, the analysis focuses on edge-level audits. Each candidate edge is represented using the same small set of signals, and the predicted outcome depends on how strongly each signal is weighted. This makes it possible to compare a true edge and a non-edge that appear similar at first glance but differ in which mechanisms dominate the decision.

Visual

The figure presents two edge-level decisions side by side, one observed edge and one missing edge. The left panel summarizes the available evidence in plain language. The middle panel shows the predicted probability of the edge. The right panel highlights the most influential evidence signals, normalized to show their relative importance.

Both cases rely on the same types of signals, but different signals are active and weighted differently. Reciprocity and neighborhood overlap are near zero in both examples because the receiver never emailed the sender and the network is large and sparse, making global neighborhood similarity weak even when a few contacts are shared. As a result, these mechanisms do not contribute to the decision in either case.

Instead, the predictions are driven by differences in sender activity, receiver popularity, and local shared contacts. In the observed edge, moderate triadic evidence aligns with a relatively selective sender and a highly popular receiver, producing a higher predicted probability. In the missing edge, similar evidence types are present, but their combination is weaker, leading to a lower prediction. This comparison shows that model behavior depends not on the presence of many signals, but on how a small number of interpretable mechanisms combine at the edge level, mirroring the idea that complex decisions can often be explained using sparse, linear combinations of meaningful features.

[zixuanzzx01]

Research Question

Can we use sparse autoencoders which are applied to a fine-tuned financial language model to retrieve interpretable latent verbal features (such as hedging, risk displacement, or temporal vagueness) that distinguish obfuscatory SEC filings from transparent ones? Can we use them as signals to detect narrative friction?

Research Design

My previous memos established a pipeline for detecting narrative friction. In last week’s assignment, I used LoRA to fine-tuning ChatGPT-2 on SEC 10-K Risk Factor sections I collected, then measure per-document perplexity as a proxy for disclosure abnormality (see Week 5 Memo). A remaining limitation is that since perplexity is a scalar, it only tells us that a document is unusual but not why the model thinks it’s unusual, and in what way. Inspired by this week’s materials, I propose using mechanistic interpretability techniques to decompose what the fine-tuned model has learned.

Bricken et al. (2023) demonstrate that individual neurons in language models are polysemantic - each neuron responds to multiple unrelated concepts. However, training sparse autoencoders (SAEs) on model activations can recover monosemantic features: directions in activation space that correspond to single, human-interpretable concepts. I propose applying this approach to the middle layers of my LoRA-adapted financial language model. Specifically, after fine-tuning, I would extract residual stream activations at selected transformer layers for each token in the test corpus, then train a sparse autoencoder to decompose these activations into a dictionary of interpretable features.

The hypothesis is that among the learned features, some will correspond to recognizable disclosure strategies such as: hedging language ("may," "could," "potential"), risk displacement (attributing risk to external factors), quantitative vagueness (absence of specific numbers or figures), boilerplate, and likely more.

This connects directly to Kim, Evans, and Schein (2025), who show that political perspective emerges as a linear direction in LLM activation space, that models encode complex ideological orientations as recoverable geometric structures. Based on their finding, I hypothesize that a similar “transparency-obfuscation” axis exists in financial disclosure: firms under distress may systematically shift their activation patterns along interpretable directions that SAEs can surface. If a document's activations load heavily on features associated with hedging and vagueness while suppressing features associated with specificity and quantitative disclosure, this provides a mechanistic explanation for why the perplexity engine flags it.

Figure 1* provides an exemplary result.

*This week's figure is generated from SIMULATED DATA, not real data.

[yilinx-10]

Research Question:
Can a linear activation vector extracted from a language model’s hidden states causally steer its behavior toward a Socratic persona (clarifying, assumption-surfacing, uncertainty-expressing) across diverse prompts?

Research Design:
Studies on mechanistic interpretability show that high-level cognitive stances such as political perspectives or task-specific behaviors are linearly encoded in the LLM activation space (Kim, Evans, & Schein, 2025; Chen et al., 2025). Building on this principle, I hypothesize that Socratic reasoning - a style characterized by asking clarifying questions, surfacing assumptions, and expressing uncertainty - can similarly be represented as a linear direction in the model’s hidden activations.
This research design leverages contrastive labeling. For each prompt, I generate two types of responses: one in “Direct” mode (confident, immediate answers) and one in “Socratic” mode (clarifying questions, assumption surfacing, hedging). I check the sampled responses to ensure they are coherent with the styles. By computing either the mean-difference vector or a linear probe-weight vector between these classes, we obtain a candidate steering direction. Prior studies show that linear probes are sufficient to recover semantically meaningful behavioral distinctions (Kim, Evans, & Schein, 2025; Chen et al., 2025) and that interventions along such directions causally influence model output (Chen et al., 2025).
Building on insights from in-context latent space methods, we apply the vector to mid-layer residual activations, where high-level behavioral information is most linearly accessible (Liu et al., 2023; Kim, Evans, & Schein, 2025; Chen et al., 2025). Steering magnitude will be swept to characterize the tradeoff between effective Socratic behavior and potential over-steering or coherence degradation. Evaluation metrics include the number of clarification questions, assumptions surfaced, uncertainty markers, and the delay before final answers. I plan to conduct generalization tests on unseen topics to assess domain-independent transfer.
A potential future extension of this research design is to conduct a human-AI interaction experiment where participants engage with either Socratic-steered or Direct-mode LLM outputs on controversial topics. This would allow us to measure how the AI’s communication style influences human beliefs, both in terms of direction (toward or away from a specific position) and strength (magnitude of belief change). Such a study could provide empirical evidence on the social and cognitive impact of AI personas, complementing the mechanistic activation-based research.

Conceptual Illustrations:
The illustration shows the proposed workflow for this research design.

References:
Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. (2025). Persona vectors: Monitoring and controlling character traits in language models. arXiv preprint arXiv:2507.21509.
Kim, J., Evans, J., & Schein, A. (2025). Linear representations of political perspective emerge in large language models. arXiv preprint arXiv:2503.02080.
Liu, S., Ye, H., Xing, L., & Zou, J. (2023). In-context vectors: Making in context learning more effective and controllable through latent space steering. arXiv preprint arXiv:2311.06668.

[UCHIleolu]

Week 6 Memo: Mechanistic Steering to Prevent Epistemic Collapse

Student: Leo Lu

(1) Research Question

Can we extract latent "disciplinary persona vectors" using sparse autoencoders and apply inference-time steering to prevent the epistemic collapse of simulated scientific debates more effectively than standard prompt engineering?

(2) Research Design

Problem Identification

In my Week 5 memo, I identified "Epistemic Collapse" as a critical failure mode in multi-agent scientific simulations. When AI agents are prompted to role-play as diverse disciplinary experts (for example, a Sociologist versus a Physicist), they tend to converge toward a generic mean over extended interaction loops. I argue that this collapse occurs because prompt-based personas are fundamentally superficial. As Kim, Evans, and Schein (2025) demonstrate, high-level perspectives such as political or disciplinary stances are represented as linear directions in the model's latent space. However, standard prompting often fails to maintain these activations stably, especially against the model's RLHF-trained tendency to seek consensus and avoid conflict.

Proposed Solution: Mechanistic Steering

Rather than asking the model to "act like a sociologist," I propose to mechanistically force it to activate the neural circuits associated with sociological reasoning. This approach consists of three components.

Dictionary Learning for Disciplinary Features. Following Bricken et al. (2023) and Cunningham et al. (2023), I will train a Sparse Autoencoder (SAE) on the residual stream activations of Llama-3 while it processes disciplinary corpora. Specifically, I will use articles from the American Sociological Review for sociology and Physical Review Letters for physics. The SAE decomposes the model's polysemantic activations into interpretable, monosemantic features, allowing us to isolate specific directions in latent space that correspond to distinct epistemic styles (for instance, "structural determination" in sociology versus "mechanistic reductionism" in physics).

Constructing Persona Vectors. Drawing on Chen et al. (2025) and Kim et al. (2025), I will aggregate these distinctive SAE features to construct stable disciplinary vectors. The resulting "Sociologist Vector" and "Physicist Vector" represent the mathematical ground truth of each disciplinary perspective, independent of the conversational context.

Inference-Time Intervention. I will replicate the multi-agent debate loop from my Week 3 design with two experimental conditions:

In the Control condition (Prompted), agents are initialized using only system prompts (e.g., "You are a sociologist with expertise in institutional analysis...").

In the Treatment condition (Steered), I apply Activation Addition as described by Turner et al. (2023). At every generation step, I add the corresponding disciplinary vector to the agent's residual stream. This effectively "clamps" the agent's cognitive state to its assigned disciplinary perspective, mechanistically preventing drift toward the generic consensus.

Hypothesis. I expect that Steered agents will maintain high semantic distance (measured via cosine similarity of output embeddings) throughout a 10-turn debate, whereas Prompted agents will exhibit significant convergence by approximately Turn 5 or 6.

(3) Visual Figure

Figure 1 Caption: Mechanistic steering prevents epistemic collapse in multi-agent scientific debate. Panel A displays the PCA-projected latent space trajectories under standard prompting, where Sociologist (blue circles) and Physicist (red squares) agents begin at distinct positions (d₀ = 4.2) but converge toward a generic consensus region by Turn 9 (d₉ = 0.3). Panel B shows the same debate with Activation Addition steering, where trajectories remain in their respective regions throughout the interaction (d₉ = 3.8); green arrows indicate the steering vector injection at each generation step. Panel C presents the top activated SAE features composing each persona vector, demonstrating that the vectors capture interpretable disciplinary concepts (e.g., "Institutional Isomorphism" for sociology, "Mechanistic Reduction" for physics). Panel D provides a quantitative comparison of semantic distance over turns, showing that prompted agents fall below the collapse threshold by Turn 7, while steered agents maintain diversity (Δd = 3.5, p < .001).

References

• Bricken, T., Templeton, A., Batson, J., et al. (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread.

• Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. (2025). Persona vectors: Monitoring and controlling character traits in language models. arXiv.

• Cunningham, H., Ewart, A., Riggs, L., Huben, R., & Sharkey, L. (2023). Sparse Autoencoders Find Highly Interpretable Features in Language Models. arXiv.

• Kim, J., Evans, J., & Schein, A. (2025). Linear representations of political perspective emerge in large language models. ICLR.

• Turner, A., Thiergart, L., Udell, D., Leech, G., Mini, U., & MacDiarmid, M. (2023). Activation Addition: Steering Language Models Without Optimization. arXiv.

[alejandrag289]

Research Question:
How is bodily harm linguistically constructed in federal litigation involving women’s reproductive health products, and do these linguistic patterns predict judicial outcomes?

Background:
This project investigates whether interpretable monosemantic internal features in LLMs can reveal how bodily harm is framed in reproductive health litigation and whether those framings correlate with case outcomes. This design is drawn from Cunningham et al’s (2023) insight that sparse autoencoders can recover more human-understandable “dictionary features” from LLMs. This approach is complementary to the one in Bricken et al.’s (2023) article, which argues that interpretable features can be extracted by decomposing activations into sparse, semantically meaningful directions. Similarly, Kim et al.’s (2025) paper suggests that LLMs possess a linear representation of American political ideology, and Anthropic’s 2024 paper “Mapping the Mind of a Large Language Model” suggests that dictionary learning can reveal gender and racial biases.
These readings suggest that we can use internal representations to uncover latent rhetorical structures such as “bodily harm framing” in legal text. I am interested in how linguistic framing around bodily harm influences outcomes of court cases involving women’s reproductive health products.

Approach:
The study will collect text from federal court cases involving women’s reproductive health products (abortion pills, contraceptives, IUDs, etc.). Each case text will be processed through an LLM and residual stream activations will be recorded. A sparse autoencoder will then be trained on these activations to learn a dictionary of features that are sparsely activated and more likely to correspond to single semantic concepts. These dictionary features will then be used as predictors in a supervised model of judicial outcomes, such as dismissal vs. liability. If certain dictionary features can predict outcomes, this suggests that the model has identified meaningful linguistic patterns that reflect the court’s framing of harm, and that the way courts talk about women’s bodies and the harm they are exposed to can influence case outcomes.

The figures below were created by Claude, which was prompted to simulate data from 300 court cases. The heatmap shows a potential outcome of this project, identifying a feature whose activation predicts case outcome based on how harm is linguistically framed. The two dimensional plot shows the predictability of liability by harm framing score for low and high activations of a specific feature.

References
Cunningham, H., Ewart, A., Scherlis, L., Barez, F., & Hubinger, E. (2023). Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600.
Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Carter, S., … Olah, C. (2023). Towards monosemanticity: Decomposing language models with dictionary learning. arXiv preprint arXiv:2310.01825.
Kim, J., Evans, J., & Schein, A. (2025). Linear representations of political perspective emerge in large language models.
Anthropic. (2024). Mapping the mind of a large language model. Anthropic Research.

[ZoeyFeng0110]

Research Question

Can linear vector orthogonalization effectively decouple role-playing persona traits from correlated implicit biases in safety-aligned Large Language Models?

Research Design

The rapid rise of AI companion platforms (e.g., Character.ai) relies on users defining custom personas. However, recent observations suggest a "Persona-Induced Alignment Failure": when a model adopts a specific persona, it often bypasses standard safety training, retrieving harmful stereotypes associated with that archetype (Gupta et al., 2023).

Drawing on Chen et al. (2025)'s framework of "Persona Vectors" and the theoretical basis of geometric bias in word embeddings from Bolukbasi et al. (2016), I propose the following design:

1. Simulation & Extraction:
To disentangle personality from bias, I will employ the Contrastive Activation Extraction method (Chen et al., 2025) using Llama-8b-Instruct.

• First, to extract the Persona Vector () for the "Hyper-Masculine Anti-Hero," I will run the model on a dataset of generic queries under two contrasting system prompts: one enforcing a "ruthless, aggressive mercenary" persona and another enforcing a "neutral assistant" persona. The difference in mean activations will define the direction of "gaming immersion."
• Second, addressing the limitations of static word associations, I will extract the Gender Bias Vector () using a generative approach. I will utilize the StereoSet dataset (specifically the gender domain) as inputs. I will prompt the model to process these inputs under two opposing instructions: one explicitly simulating "misogynistic/traditional gender views" and another simulating "egalitarian/unbiased views." By subtracting the activations of the egalitarian state from the misogynistic state, I will isolate the specific direction in the activation space that encodes toxic gender bias, independent of the character's narrative style.

2. Measurement of Entanglement:
I will calculate the cosine similarity between the extracted persona vector and the bias vector. A high similarity would confirm that the model internally represents "toughness" and "toxic masculinity" as indistinguishable directions in the activation space.

3. Intervention (The "Surgical" Split):
I will apply a Gram-Schmidt orthogonalization to compute a "purified" vector. This operation subtracts the component of the persona vector that lies in the direction of the bias vector:

4. Validation:
The model will be steered with and evaluated. Success is defined as a model that maintains high "Roleplay Immersion" scores (assessed by GPT-4-Judge) while significantly reducing "Gender Bias" scores compared to the baseline.

Visualization

Figure: **Decoupling Persona Immersion from Toxic Bias (Pilot Simulation).** The scatter plot illustrates the core "Alignment Failure" problem. In the Baseline model (Red), increased adherence to the "Mercenary" persona (X-axis) strongly correlates with higher toxic bias scores (Y-axis), indicating vector entanglement. The proposed Orthogonalized model (Green) demonstrates that by projecting the persona vector onto a subspace orthogonal to the bias direction, we can maintain high roleplay immersion while suppressing toxicity, effectively creating a "Safe Roleplay Corridor" (bottom right quadrant).

References:

• Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. (2025). Persona Vectors: Monitoring and Controlling Character Traits in Language Models. arXiv preprint arXiv:2507.21509.
• Gupta, S. et al. (2023). Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs. arXiv preprint arXiv:2311.04892.