Fix LiteLLM Installation Errors from puccinialin Dependency (Use tokenizers== 0.14.1)

[dev-wjood]

Hi everyone! 👋

I recently encountered an installation issue with LiteLLM where pip install litellm failed due to an unresolved dependency called puccinialin. After some digging, I discovered this stems from a problematic version of the tokenizers package.

The Problem:

Newer versions of tokenizers (likely >=0.21.0) reference puccinialin, which doesn’t exist on PyPI.
This causes LiteLLM installation to fail with:

ERROR: Could not find a version that satisfies the requirement puccinialin

The Solution:

1. Install LiteLLM without dependencies:

pip install litellm --no-deps

2. Manually install tokenizers and core dependencies:

pip install \
  aiohttp \
  click \
  "httpx>=0.23.0" \
  "importlib-metadata>=6.8.0" \
  "jinja2>=3.1.2,<4.0.0" \
  "jsonschema>=4.22.0,<5.0.0" \
  "openai>=1.68.2" \
  "pydantic>=2.0.0,<3.0.0" \
  "python-dotenv>=0.2.0" \
  "tiktoken>=0.7.0" \
  "tokenizers==0.14.1"

Tested Environment:

Python 3.8
macOS

Note:

If other dependencies are missing, refer to LiteLLM's pyproject.toml and install what mentioned under dependencies section.