Skip to content

ModernBertEmbedder hardcodes mean pooling; ignores 1_Pooling/config.json #212

Description

@amgix-dev

Problem

ModernBertEmbedder always uses mean pooling, regardless of the model's Hugging Face config.

In rust/src/embeddings/local/modernbert.rs, pooling is hardcoded at load time:

Ok(ModernBertEmbedder {
    model,
    tokenizer,
    device,
    pooling: Pooling::Mean,  // always mean
})

Many ModernBERT embedding models ship with 1_Pooling/config.json specifying CLS pooling, for example:

  • ibm-granite/granite-embedding-97m-multilingual-r2pooling_mode_cls_token: true
  • Alibaba-NLP/gte-modernbert-basepooling_mode_cls_token: true

Using ModernBertEmbedder::embed() on these models ignores the model's declared pooling strategy.

Expected behavior

On load, resolve pooling from the model repo:

  1. If 1_Pooling/config.json exists, use it (pooling_mode_cls_token, pooling_mode_mean_tokens, or pooling_mode)
  2. Else fall back to config.jsonclassifier_pooling ("cls" / "mean")
  3. Else default to mean

BertEmbedder already accepts pooling: Option<Pooling> in new(); ModernBertEmbedder should follow the same pattern.

Suggested fix

  • Add pooling: Option<Pooling> to ModernBertEmbedder::new
  • Add a small helper to load pooling from the HF repo (1_Pooling/config.json + fallback)
  • Pass pooling through from TextEmbedder::from_pretrained_hf (currently ignored for ModernBERT)

Impact

Any user calling ModernBertEmbedder::embed() on CLS-pooled models gets incorrect embeddings.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions