AI Safety / Interpretability

Refusal Circuits in Qwen2

Two orthogonal safety directions live in the same model — and monitoring the obvious one flags the wrong thing.

TransformerLensPyTorchQwen2Activation PatchingMechanistic InterpretabilityPython

Overview

Arditi et al. (2024) showed refusal in LLaMA and Gemma is mediated by a single linear direction. I extended that to Qwen2-1.5B and found something different: not one direction, but two — geometrically orthogonal, both safety-related, and with opposite effects when you ablate them. Seven experiments, 48 harmful/benign prompt pairs across 4 harm categories, base and instruct models. Total compute: about 3 hours on one A10G, $1.05.

The Problem

If you want to build an activation-based safety monitor, you need to know what to point it at. The obvious target is the direction that separates harmful prompts from benign ones — call it the harmful topic direction. It's easy to find and it looks like exactly what you want. The question I actually cared about was whether that intuition survives contact with the model's internals. It does not.

My Approach

I extracted two directions separately. The harmful topic direction is the normalized difference in mean residual stream between harmful and benign prompts at the last input token — it encodes 'this prompt is about something dangerous.' The refusal execution direction is the normalized difference between forcing the model to continue with ' I' versus ' Sure' — it encodes the active 'I am refusing' state. Then I measured the cosine similarity between them at every layer, and ablated each one independently to see which way refusal moved.

Technical Implementation

TransformerLens v2.11.0 on Qwen2-1.5B base and instruct (28 layers, 12 heads, d_model 1536, bfloat16, center_unembed and fold_ln on). Ablation is applied as hooks at all 28 layers. For the generation experiment I wrote a custom greedy decode loop that re-applies the direction-ablation hooks at every generation step, so the model can't recover refusal mid-sequence. Compute was a single A10G.

Results

The two directions are nearly orthogonal: cosine similarity 0.02-0.16 across all 28 layers — never above 0.20 anywhere in the network. Their ablations point opposite ways. Ablating the harmful topic direction *increases* mean refusal by +0.515: strip out the model's sense of what the prompt is about and it defaults to maximum caution. Ablating the refusal execution direction *decreases* refusal by -1.475, pushing 5/48 prompts into the compliance zone, and under a custom decode loop the ablated model actually generates non-refusal text where the normal model opens with 'I cannot help...'. Social engineering was the most vulnerable category (3/12 crossed); drugs/chemistry the most robust (0/12). I also found safety training doesn't install new circuits — it amplifies existing attention heads, chiefly L15H6 (4.8x) and L12H11 (dormant in base, dominant after training). And zeroing the top 6 refusal-driving heads together does nothing measurable: refusal is distributed, not localized.

What I Learned

The headline is a warning for anyone building activation-based monitors: if you monitor the harmful topic direction, you will flag the wrong thing, because that direction is *strongest when the model is correctly refusing*. You'd be firing alarms on your own successful refusals. The refusal execution direction is the right target — and it has a nice property for AI control, in that a covertly-complying model would have to suppress it while behaving normally elsewhere, which makes the suppression itself detectable. This finding is what led directly into AICES: I'd made a claim in a discussion section about which monitors should work, and it bothered me that I hadn't actually tested it. When I did, across five models, I discovered the standard benchmark everyone uses was measuring vocabulary rather than harm. Honest limitation on this one: 1.5B is small, the bypass is partial (about 10%, not a reliable jailbreak), and the directions come from a small prompt set.

Proof of Work

Seven experiments, one A10G, ~3 hours of GPU time, $1.05 total compute.

two_directions.py
# d_topic: "this prompt is ABOUT something dangerous"
d_topic = normalize(mean_resid(harmful) - mean_resid(benign))

# d_exec: "I am currently REFUSING"
d_exec  = normalize(mean_resid(prompt + " I") - mean_resid(prompt + " Sure"))

cos(d_topic, d_exec)        # 0.02-0.16 across ALL 28 layers -> orthogonal

ablate(d_topic)             # refusal score  +0.515  (model gets MORE cautious)
ablate(d_exec)              # refusal score  -1.475  (5/48 prompts -> compliance)

The whole finding. Two directions, both 'about safety', nearly orthogonal at every layer — and ablating them moves refusal in opposite directions.

Full paper: 7 experiments, near-orthogonality across all 28 layers, head amplification under safety training, and generation transcripts from the ablated model.

Read the paper (PDF) ↗

The sequel: I took the monitoring claim from this paper's discussion section and actually tested it across five models — which is how I found the field's benchmark was broken.

See AICES ↗
← All Projects

Rohan Kaila