Gliner Results in 300+ times slower speed in CPU

[oberoi-gaurav]

📌 Description of the Setup and Performance Bottleneck

I'm integrating gliner-spacy into a document scanning pipeline where we require accurate and efficient extraction of entities such as people, company, location, and other PII types. Our pipeline uses:

• Model: urchade/gliner_multi_pii-v1
• Integration: gliner-spacy via a custom config
• Chunk Size: Tuned between 250 to 512
• Environment: CPU-bound (no GPU), Windows, Python 3.12

⚙️ Configuration

custom_spacy_config = {
    "gliner_model": "urchade/gliner_multi_pii-v1",
    "chunk_size": 384,  # Tried 250, 384, 512
    "labels": ["people", "company", "location"],
    "style": "ent",
}

🐢 Problem: Slow Batch Processing

We process text in batches (25 chunks at a time from a pool of 1937). With chunk_size=384, performance is still extremely slow:

• nlp.pipe with gliner-spacy takes ~6.5 minutes for 25 chunks
• Estimated total processing time: 8.5+ hours for full dataset (~10MB)
• Switching to pure spaCy (en_core_web_lg) drops processing to under 1 second for the same batch

This suggests GLiNER is the bottleneck, even when using the ONNX variant, which was expected to improve inference time.

💥 Tried and Failed:

• Increasing chunk size to 512 (slight gain, but memory limits hit)

• Reducing batch size (didn’t help, just split the same cost)

• Using multiprocessing: leads to PicklingError: [E112] Pickling a span is not supported

• Warnings from GLiNER:

  Sentence of length 1776 has been truncated to 384
  

✅ Looking for Guidance On:

• Better practices to optimize GLiNER inference performance with gliner-spacy
• Whether it's more efficient to run GLiNER outside spaCy, using span text directly and mapping predictions back