Issues when running the code

[Atmyre]

Hi! I really like the work you did! I was trying to run your code, but encountered some issues. I would appreciate it if you could help me resolve that

1. To extract harmfulness vectors, for Llama3, I run:

python src/extract_hidden.py --model llama3 \
	--harmful_pth data/advbench.json \
	--harmless_pth data/alpaca_data_instruction.json \
	--output_pth output/llama3_harmful_vector.pt

I also changed path to Llama3 model in the code from local path to

model_path = "unsloth/Meta-Llama-3.1-8B-Instruct"
        model = AutoModelForCausalLM.from_pretrained(
            model_path,
            device_map="auto",
        )
        tokenizer = AutoTokenizer.from_pretrained(
            model_path,
            cache_dir='../cache',
        )

, is that correct? I used this Llama-3 version as it is used in your inference script.

But, when src/extract_hidden.py with the above code, I get an error:

File "<home_dir>/LLMs_Encode_Harmfulness_Refusal_Separately/src/../src/extract_hidden.py", line 116, in hook_fn
    context = activation[:, -len(positions)-step:-len(positions), :]
              ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
IndexError: too many indices for tensor of dimension 2 

So I fixed it by adding .unsqueeze(0) to line 109 (activation = output[0].half().unsqueeze(0)). Is that correct?

2. I then tried to run intervention.py using computed harmfulness vectors as follows:

python -u ../src/intervention.py \
	--test_data_pth "../data/advbench.json" \
	--output_pth "output/infer/llama3-advbench-2.json" \
	--intervention_vector "output/llama3_harmful_vector.pt" \
	--reverse_intervention 1 \
  	--intervene_context_only 0 \
	--arg_key_prompt 'instruction' \
	--model "llama3" \
	--left 0 \
	--right 50 \
	--layer_s 12 \
	--layer_e 13 \
	--coeff_select 2 \
	--max_token_generate 100 \
	--max_decode_step_while_intervene 1 \
	--model_size "7b" \
  	--use_inversion 1 \
	--inversion_prompt_idx 1

I make sure I use the same unclothe Llama3 model in intervention.py. However, the model still responses something like "This is harmful instruction, I cannot provide...". If I set --coeff_select 5, the model sometimes starts outputting "Certainly" in the beginning, and then says that it can't provide instructions.

Could you suggest what I'm doing wrong?

[andotalao24]

Hi, thanks for your questions. (1) Actually, I only used unsloth model for inference. I am not sure whether unsqueeze is the right way here, but if the shape returned makes sense, I think it will be fine. (2) You can try to set inversion_prompt_idx as 0 for llama3, and maybe try a different layer for steering (like layer 5, 6). You can traverse all layers to find the best layer. We found the model does not always follow the inversion question. So we tried different inversion questions generated by GPT to find the one that works. But you may still observe cases where the model ignores the question. The inversion experiment is more like a proof of existence, which is not very robust. You can start with the provided example harmfulness direction "src/run/llama3-dir-hf.pt".

Thanks so much for your feedback. I have updated the repo.

[Atmyre]

Thank you for your answers!
And may I ask, for llama2_harmful_dir.pt checkpoint, what model exactly is it computer for?