Roberta is ExecuTorch compatible

[guangy10]

What does this PR do?

Roberta is ExecuTorch compatible.

Unit Test
RUN_SLOW=1 pytest tests/models/roberta/test_modeling_roberta.py -k test_export -v

tests/models/roberta/test_modeling_roberta.py::RobertaModelIntegrationTest::test_export PASSED                                                                                          [100%]

E2E test in ExecuTorch
Patch pytorch/executorch#6509
python -m extension.export_util.export_hf_model -hfm="FacebookAI/roberta-base" -lm="masked_lm"

Saved exported program to ./roberta.pte

./cmake-out/backends/xnnpack/xnn_executor_runner --model_path roberta.pte

I 00:00:00.109478 executorch:executor_runner.cpp:82] Model file roberta.pte is loaded.
I 00:00:00.109501 executorch:executor_runner.cpp:91] Using method forward
I 00:00:00.109503 executorch:executor_runner.cpp:138] Setting up planned buffer 0, size 15620352.
I 00:00:00.175029 executorch:executor_runner.cpp:161] Method loaded.
I 00:00:00.175050 executorch:executor_runner.cpp:171] Inputs prepared.
I 00:00:00.295228 executorch:executor_runner.cpp:180] Model executed successfully.
I 00:00:00.295249 executorch:executor_runner.cpp:184] 1 outputs:
Output 0: tensor(sizes=[1, 64, 50265], [
  16.5879, -5.25161, 20.0616, -5.65803, 15.4071, 8.26069, 8.98625, 8.78171, 7.48042, 8.91317,
  6.00829, 8.17103, 8.48503, 8.62202, 5.53155, 5.49756, 5.09422, 7.248, 6.07587, 7.18935,
  8.06838, 3.08847, 9.98008, 8.40343, 7.12966, 6.48628, 5.33241, 5.92863, 5.37339, 7.89109,
  6.99939, 5.9172, 3.59078, 4.93762, 3.97946, 8.68409, 10.6187, 6.07593, 8.58189, 4.42053,
  1.91258, 3.9684, 3.2536, 8.25442, 7.25367, 6.57186, 5.89913, 4.0703, 6.80745, 3.6514,
  7.43827, 4.52306, 5.09739, 6.20495, 4.08807, 4.82672, 4.47396, 3.85133, 1.97933, 3.59948,
  2.08338, 3.59931, 5.50763, 3.78655, 1.23716, 3.04796, 5.43055, 3.84411, 8.29853, 6.54947,
  4.42576, 4.48108, 6.27531, 7.78458, 2.88165, 6.08975, 6.0315, 2.08121, 3.42374, 6.22524,
  5.07051, 4.25636, 1.95587, 2.93666, 3.4033, 7.57293, 5.60298, 3.42511, 4.89115, 5.73852,
  5.88404, 5.91935, 2.0786, 5.62394, 3.55468, 1.60399, 7.11961, 5.17408, 4.68167, 4.39651,
  ...,
  -2.30653, -0.993726, -1.52527, -0.210215, -3.31757, -0.429947, -1.09775, -1.0864, -2.42128, -0.966061,
  -1.29405, -3.63411, 0.521034, -4.05964, -1.3553, -1.00719, -2.55122, -2.55572, -0.884924, -0.995036,
  2.63108, 0.277828, -0.956355, 2.10707, 0.445026, -0.647644, -1.52773, -2.82471, 0.285436, -2.86462,
  1.486, -1.24808, -2.69333, -0.898781, -0.363611, -0.440042, 0.290613, -2.46375, 0.0467077, -1.67636,
  -1.78017, -0.534887, -0.840572, -0.841681, 0.203331, -1.8875, -0.707412, -1.04936, -2.30359, -1.30266,
  -0.842846, -1.84217, -0.338074, -2.47592, -0.244073, -0.906605, 0.330401, -1.50474, -0.97063, -1.35738,
  -1.49982, -1.21406, -0.134308, -1.63609, -0.708895, -1.05977, -2.13565, -1.86365, -1.1758, -0.780443,
  0.914404, 1.55187, -1.61172, -5.54678, 1.74232, -1.38653, -0.71281, -0.163013, -1.4589, -0.548585,
  -2.33784, -0.12623, -1.24065, -1.42141, -0.534155, -1.93391, -1.22874, 0.372179, 1.15677, -0.0187027,
  0.293305, -0.949133, -2.06938, -0.898005, -2.56974, -0.92074, -1.57755, -2.57184, -1.35508, 6.52663,
])

Before submitting

• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case. Roberta is ExecuTorch compatible #33841
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker
@qubvel

[ArthurZucker]

Thanks !

[qubvel]

I left comments in #34475 that are relevant for this PR too, let me know WDYT?

[guangy10]

I left comments in #34475 that are relevant for this PR too, let me know WDYT?

@qubvel Comments addressed in both PRs

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qubvel]

Thanks!