How to attack flan-t5 or ChatGPT models

[CristianCosci]

Hi, as i understood, the attack can be made only on "classifier" models that output numbers. How can i re-write some code part, and which of them, in order to make attack on model that outputs text? For example the base versione of flan-t5-large, chatGPT via API or LLaMa model?

In particular i want to know where to rewrite the inference phase of the model to transoform text in label.

To be more specific, if my model make the classification like this:

model = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-base").to("cuda")
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-base")


def inference(*, prompt, sentence, tokenizer, model):
    inputs = tokenizer(prompt(sentence), return_tensors="pt").to("cuda")
    outputs = model.generate(**inputs, max_new_tokens=50)
    output = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]

    if "NEGATIVE" in output:
        return 0
    elif "POSITIVE" in output:
        return 1
    else:
        #print("Model is confused... generated text: ", output)
        return 0

print(inference(prompt=prompt, sentence="I love you", tokenizer=tokenizer, model=model))

Where have i to insert the model loading and inference?