Skip to content

Commit 6023f6e

Browse files
committed
Add conversion method to config in gpt-oss architecture adapter
1 parent a6bddfa commit 6023f6e

File tree

1 file changed

+6
-1
lines changed
  • transformer_lens/model_bridge/supported_architectures

1 file changed

+6
-1
lines changed

transformer_lens/model_bridge/supported_architectures/gpt_oss.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,12 @@ def __init__(self, cfg: Any) -> None:
4747
"experts": BlockBridge(
4848
name="experts",
4949
submodules={
50-
"gate_up": JointGateUpMLPBridge(name="gate_up_proj"),
50+
"gate_up": JointGateUpMLPBridge(
51+
name="gate_up_proj",
52+
gate_up_config={
53+
"split_gate_up_matrix": self.split_gate_up_matrix
54+
},
55+
),
5156
"down": LinearBridge(name="down_proj"),
5257
},
5358
),

0 commit comments

Comments
 (0)