Mac runners segmentation fault on pin updates

[jackzhxng]

The pin updates in #117 cause a segmentation fault / bus error on all mac tests when running forward on the loaded pte, regardless of model size. A few folks have taken a look but we'll likely need someone to dive in deep to figure this one out.

Example segmentation fault error: https://github.com/huggingface/optimum-executorch/actions/runs/16973422485/job/48116076576
Example bus error: https://github.com/huggingface/optimum-executorch/actions/runs/16973422485/job/48116076175

The mac test have current been commented out, this issue is to fix the model forward error and re-enable the tests.

[digantdesai]

Tried RUN_SLOW=1 pytest tests/models/test_modeling_dit.py -s -vvvv --durations=0 --log-cli-level=INFO locally with 40b02a2dc61bbf901a2df91719f47c98d65368ec which uses ET 0.7.0 on my M1, and it worked OK.

[digantdesai]

RUN_SLOW=1 pytest tests/models/test_modeling_gemma3.py -s -vvvv --durations=0 --log-cli-level=INFO This also passed both XNNPACK and Portable variants locally on my M1.