Memory issues with HookedTransformer #5
Description
Hello! I really like your paper and I am doing similar research currently. I would like to know if you have experienced issues with evaluating models loaded via HookedTransformer? I found that models loaded via HookedTransformer consumed too much memory so I was getting OOM errors during first iterations of chatting with the model. In addition, I was also interested what is the reason behind using lm-evaluation-harness library for evaluation? Thanks in advance!