Imlement tests for abstract structures such as in Curcuits thread

[oserikov]

duration: scalable, can be both 175 and 350 hours
mentor: @oserikov , Ryan Teehan
difficulty: medium
requirements:

1. PyTorch
2. sklearn
3. experience with re-using the academic code
4. experience with Transformer Language models

useful links:

• The Circuits thread
• A Primer on Bertology

Idea Description:

In Circuits, several abstract structures found in CV models were summarized. The Branches Specialization tendency of the CV neural networks, as well as the Weight Banding property of NNs last layers, have not been directly studied in LLMs, though the findings of several papers (1, 2) could be related.

The task is to perform a study of the abstract structures representedness in CV and NLP models, by applying the same inspection techniques to both groups of models. This would allow answering the following questions. Are the abstract structures identified for CV also found in NLP models? How do these structures differ among NLP models?

Coding Challenge

Reproduce the Branch Specialization core insight: demonstrate the presence of the branch specialization in the CV model; Reproduce the Individual Neurons analysis on BERT model.