Revert PEP 604 union syntax to Union[]/Optional[] for Python 3.9 compatibility

[Copilot]

What does this changes

Converts all PEP 604 union type syntax (X | Y) to traditional Union[X, Y] and Optional[X] across 46 files.

What was wrong

PEP 604 syntax (str | None, list[str] | str) requires Python 3.10+. While from __future__ import annotations allows the syntax to parse in 3.9, it stores annotations as strings and can cause runtime failures in type checking or introspection. The project's requires-python = ">=3.9" demands full compatibility.

How this fixes it

• X | None → Optional[X]
• X | Y → Union[X, Y]
• X | Y | Z → Union[X, Y, Z]

Added typing imports (Union, Optional) to affected modules, merged with existing imports alphabetically. Preserved from __future__ import annotations for forward compatibility.

Example transformation:

# Before
def dependency_parsing(
    text: str,
    model: str | None = None,
) -> list[list[str]] | str:

# After
from typing import List, Optional, Union

def dependency_parsing(
    text: str,
    model: Optional[str] = None,
) -> Union[List[List[str]], str]:

Affected modules: tokenize, parse, translate, spell, summarize, tag, transliterate, util, corpus, and 10+ others.

Your checklist for this pull request

• Passed code styles and structures
• Passed code linting checks and unit test

Original prompt

Revert all the PEP 604 (Union Types as X | Y) to Union[] (or Optional[], for | None), to keep Python 3.9 support.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[sonarqubecloud]

Quality Gate passed

Issues
5 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[coveralls]

coverage: 53.556% (-0.06%) from 53.615%
when pulling 273c396 on copilot/revert-pep604-union-types
into 9e6520b on dev.