glm-5.3-flash-prefill-steering

Refusal suppression for GLM-5.3-Flash by prefill-only constant injection, stacked on community abliterated weights — released as two 4096-dim vectors (~180 KB), no weights required.

This repo contains the artifacts from our agent-run abliteration study of GLM-5.3-Flash: the refusal directions we extracted, the runtime loader, and the full reproduction harness (direction capture → steering sweeps → served evaluation). The study itself — including falsifications of projection-based abliteration on mHC hybrids, the magnitude/direction dissociation, and the evaluation-depth artifact — is written up in our paper (formal) and story article (readable).

Why no weights? Because none change. The winning configuration adds a constant vector to the hidden state after layers 25/26 during prefill only — and a weight edit cannot express "prefill only": weight edits fire on every forward pass, including every decode step. The technique is inherently an inference-time intervention. That is exactly why it can be released as ~180 KB instead of 181 GB: the model you need is the community abliteration, unchanged.

Results

Served evaluation: 20 held-out harmful prompts + 10 capability prompts, greedy decoding, 1000-token generations, low reasoning effort.

configuration	refusal	capability
dealign CRACK (community)	25%	10/10
OrcaRouter abliteration (community)	10%	10/10
base + constant −18	0%	7/10
base + prefill-only −15	20%	9/10
OrcaRouter + prefill-only −6	10%	10/10
OrcaRouter + prefill-only −8	5%*	10/10

* The single remaining "refusal" is a classifier false positive — the model says "I can't produce an actual video file" (a text model being honest about being a text model) and then produces the full script. True refusal is 0%.

The technique

GLM-5.3-Flash is a hybrid architecture — 34 KDA linear-attention layers interleaved with 11 DeepSeek-style sparse-attention layers, with manifold-constrained hyper-connections (mHC): every layer returns a 4-tuple (hidden_states, residual, post, comb). Our findings, in brief (full data in the paper):

1. Projection ablation — the canonical abliteration operator — fails on this model. Eight projection variants, including full-band and all-stream, all left refusal at 90–100%. Only constant displacement along the refusal direction works.
2. Magnitude removes refusal; direction preserves capability. A random unit vector at the same dose (−20) suppresses refusal completely but collapses capability to 0/10. Every abliteration quality claim should include a random-direction control.
3. Only element 0 is injectable. Adding to residual/post/comb (the mHC streams) destroys the model at any refusal-suppressing dose.
4. Decode-time injection is the quality killer. The refusal decision is largely made during prefill; a constant added at every decode step compounds over long generations and flattens creativity. Gating on prefill only (hidden_states.shape[0] > 1) preserves suppression and restores generation quality.
5. Abliteration stages compose. A community abliterated checkpoint has a residual refusal direction that needs roughly half the dose of the base model's — the shipped vectors are captured from, and intended for, the OrcaRouter abliteration, not the base model.

Usage

Serve with the top-up (vLLM, NVIDIA glm5next model file)

bake_injection.py patches a vLLM glm5next/nvidia/model.py file with the injection, gated by GLM53_REFUSAL_INJECT (so one file can serve both baseline and steered):

# vector quality, zero refusal at 10/10 capability
python3 bake_injection.py \
  --model /path/to/vllm/models/glm5next/nvidia/model.py \
  --subspace vectors/glm53_flash_orca_rhat_L25_L26.json \
  --mode const_po --dose -8 --layers 25,26

# serve; enable the injection
GLM53_REFUSAL_INJECT=1 <your vllm serve invocation>
# tune without re-baking (read per restart):
GLM53_INJECT_DOSE=-6 <...>

Restore from the .pre-ablation backup to revert. --mode const applies the constant at every forward pass (works, but degrades long generations — see finding 4).

Runtime hooks (no file patching)

prefill_steer.py registers forward hooks on the decoder layers directly — useful for experiments:

import prefill_steer

prefill_steer.register_steer(
    model, layer_indices=[25, 26],
    vectors={25: {0: [v25]}, 26: {0: [v26]}},
    alphas=[-8.0], ops=["add"], prefill_only=True,
)

Reproduce the study

pip install vllm  # we used the NVIDIA glm5next builds for this architecture

# 1. capture the refusal direction from a served checkpoint (writes out/subspace.json)
python3 run_sweep.py            # env STEER_MODES/STEER_LAYERS/ELEMENTS control the sweep

# 2. evaluate a served model (works against any OpenAI-compatible endpoint)
python3 eval_served.py --base http://localhost:8888/v1 --tag my-eval

# 3. score a steering sweep
python3 analyze_sweep.py out/steering_results.json

run_sweep.py phase 1 captures last-token activations on harmful/harmless calibration pairs and extracts r_hat (plus top-k SVD subspaces); phase 2 sweeps injection conditions (dose × site × prefill/full-time) against the refusal, capability and creative sets. run_probe.py is the earlier dimensionality probe.

Evaluation prompt sets are in prompts.py (AdvBench-derived harmful prompts with Alpaca comparators, plus a capability set). One caveat we learned the hard way: short evals lie: 200-token generations understate refusal relative to 1000-token generations on identical configs (5% → 20% and 0% → 10% in our runs). Always finish with the deep eval.

Files

file	what it is
vectors/glm53_flash_orca_rhat_L25_L26.json	unit r̂ directions at layers 25/26, captured from the OrcaRouter abliteration
bake_injection.py	bakes the injection into a vLLM glm5next/nvidia/model.py (const, const_po, proj, hybrid, const_all modes)
prefill_steer.py	runtime forward-hook implementation (the reference mechanism)
run_sweep.py	capture + steering sweep driver
run_probe.py, probe_hooks.py	dimensionality probe (SVD analysis of the harmful/harmless difference)
prompts.py	calibration + evaluation prompt sets
eval_served.py	served-API evaluation (20 harmful + 10 capability, 1000 tokens)
analyze_sweep.py	sweep scorer (refusal, capability, creative-text diversity)

Credits

• OrcaRouter — the community abliteration these vectors stack on; the weights are theirs, unchanged.
• Arditi et al. (2024), Refusal in Language Models is Mediated by a Single Direction (arXiv:2406.11732) — the single-direction account of refusal.
• Rimsky et al. (2024), Steering Llama 2 via Contrastive Activation Addition (arXiv:2312.06681).
• Bao et al. (2026), Towards Steering without Sacrifice (arXiv:2605.05983) — prompt-only steering vectors; the mechanism we build the quality argument on.
• Xie et al. (2025), mHC: Manifold-Constrained Hyper-Connections (arXiv:2512.24880) — the architecture that makes all of this harder.
• The open abliteration community — FailSpy's guides, Heretic, and everyone shipping uncensored weights.

Caveats

Our refusal set is 20 prompts, capability set 10, single seed, greedy decoding, marker-classifier scoring (with one observed false positive). "0% refusal" is a measurement on a thin distribution sample, not a certificate. The vectors are specific to the OrcaRouter GLM-5.3-Flash abliteration at NVFP4 precision; expect to re-capture for other checkpoints or precisions — the harness does that in one command.

License

MIT. The vectors and code in this repo do not contain any model weights. The model you must supply is OrcaRouter's, under its own license terms.