The conditionals for xetex & luatex inherited from quarto partials appear to trigger arXiv to 'auto-detect' requirement for XeTex or LuaTex, and consequently reject the submitted files as shown below.
The work around for this is to overwrite the font.latex and font-settings.latex partials and remove any of the conditional references to xetex/luatex packages: