Albert is ExecuTorch compatible

[guangy10]

What does this PR do?

Albert is ExecuTorch compatible.

Unit Test:
RUN_SLOW=1 pytest tests/models/albert/test_modeling_albert.py -k test_export -v

tests/models/albert/test_modeling_albert.py::AlbertModelIntegrationTest::test_export PASSED                                                                                             [100%]

E2E test in ExecuTorch:
Patch pytorch/executorch#6509
python -m extension.export_util.export_hf_model -hfm="albert/albert-base-v2" -lm masked_lm

Saved exported program to ./albert.pte

./cmake-out/backends/xnnpack/xnn_executor_runner --model_path albert.pte

I 00:00:00.051666 executorch:executor_runner.cpp:82] Model file albert.pte is loaded.
I 00:00:00.051701 executorch:executor_runner.cpp:91] Using method forward
I 00:00:00.051704 executorch:executor_runner.cpp:138] Setting up planned buffer 0, size 12005376.
I 00:00:00.101731 executorch:executor_runner.cpp:161] Method loaded.
I 00:00:00.101775 executorch:executor_runner.cpp:171] Inputs prepared.
I 00:00:00.251130 executorch:executor_runner.cpp:180] Model executed successfully.
I 00:00:00.251148 executorch:executor_runner.cpp:184] 1 outputs:
Output 0: tensor(sizes=[1, 64, 30000], [
  7.12213, 16.4255, -9.69697, 0.315882, 7.49277, 8.37929, 8.01692, 12.2838, 8.11998, 12.4227,
  7.5468, -5.25646, -5.68964, 11.3917, 8.85877, 8.94538, 5.69543, 7.87437, 10.1869, 6.47921,
  5.09051, 8.5898, 7.79427, 1.2211, 3.30417, 3.22097, 1.58806, 9.30696, 1.07529, 4.84525,
  2.17895, 8.81211, -1.02848, -3.64258, 6.78737, 4.30354, 1.65078, 3.47092, 11.7028, 7.89638,
  5.70505, -1.05684, 8.3248, 12.2657, 4.26686, 10.2256, -1.99968, 2.86684, 1.18797, 16.2016,
  1.63196, 5.46712, 2.33064, 7.08936, 0.676241, 6.57334, 1.04902, 0.281277, 12.6735, -1.04131,
  4.93435, -5.3161, 10.982, 2.07643, -1.98044, 1.17825, -6.78902, -0.594365, 9.06238, 11.7988,
  6.41249, 2.30598, 2.37509, 8.14539, 0.708781, 0.270195, -0.437858, -3.87035, -3.94704, 12.5791,
  0.291936, 5.41188, -2.38334, -4.61858, 2.57807, -0.0342076, -2.09207, 3.3832, 4.2705, -5.35976,
  6.55041, -5.35834, 0.0824419, 10.0817, -11.5175, 7.71341, 14.2482, -2.19647, 0.258341, 13.5795,
  ...,
  -26.6734, -15.8391, -9.05885, -22.9564, -14.1135, -14.3582, -1.38681, -22.967, -6.46937, -5.23052,
  -15.8735, -0.781886, 1.96928, -0.801466, -13.4606, -9.3534, -7.63344, -18.6456, -14.0491, -10.0933,
  -10.3132, -11.3254, -12.3537, -4.23457, -9.51285, -19.6473, -14.6648, -5.87785, -2.96578, -14.0239,
  -0.557438, -5.21334, -5.5204, 1.0429, -8.47772, -12.8267, -8.01721, -15.3659, -15.7359, -14.8388,
  -11.0749, -14.5002, -22.6418, -7.16905, -5.90876, -12.3513, -9.51316, -21.9345, -18.6938, -6.07597,
  -11.0177, 0.404317, -6.31417, -8.48093, -7.75292, -7.26334, -14.5192, -9.14845, -10.1494, -5.35306,
  -2.2068, -12.4971, -20.1255, -3.67846, -8.99902, -11.6741, -13.5727, -0.0831118, -12.0526, -7.48546,
  -18.1656, -12.0559, -6.95208, 1.14825, -11.53, -13.2759, -9.91268, -8.80736, -3.15759, -4.27456,
  -6.43947, -7.06724, -8.69398, -13.4397, -4.94796, -9.45768, -11.02, -11.3739, -10.9547, -18.7554,
  -25.8251, -12.1951, -6.00279, -9.81018, -5.64514, -20.6445, -12.1152, -7.1209, -13.5729, -8.33296,
])

Before submitting

• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case. Albert is ExecuTorch compatible #33836
• Did you write any new necessary tests?

Who can review?

@ArthurZucker
@qubvel

[ArthurZucker]

Thanks 🤗