Where is the step 'agent selection' and 'team reformation' in llmlp_listwise_mmlu.py

[ME-Msc]

DyLAN/code/MMLU/llmlp_listwise_mmlu.py

Lines 45 to 58 in 006e440

	for que, ans in qa_pairs:
	llmlp.zero_grad()
	res, resp_cnt, completions, prompt_tokens, completion_tokens = llmlp.forward(que)
	imp_score = llmlp.backward(res)
	completion_list.append(completions)
	accs.append(ans == res)
	resp_cnts += resp_cnt
	importances.append(imp_score)
	total_prompt_tokens += prompt_tokens
	total_completion_tokens += completion_tokens
	with open(DIR_NAME+'/'+EXP_NAME+'_'+str(len(ROLES))+'3.json', 'a') as f:
	f.write(json.dumps(completions) + '\n')

As the title describes, I didn't see the step 'agent selection' and 'team reformation' in llmlp_listwise_mmlu.py lines 45-58. You only did forward (propagation in section 3.4(1) of your paper) and backward (aggregation in section 3.4(2) of your paper).

Could you update the code? Thanks.

[zhipeng-wei]

I also noticed that no optimization steps are included.

[ME-Msc]

I think it's the old version code. They didn't implement the experiment as the paper said. Fake research.

[permutation248]

I think it's the old version code. They didn't implement the experiment as the paper said. Fake research.

绷不住了

[Tsai-ky]

The workflow of this work is quite confused...

[fmxu2016]

沽名钓誉之徒啊