Add support for BertForSequenceClassification reranking

[huydt84]

This PR includes:

• Addition to convert_hf_to_gguf to handle BertForSequenceClassification model
• Fallback to [SEP] token if [EOS] didn't exist (for BertForSequenceClassification)
• Make llm_graph_context::build_pooling work with BertForSequenceClassification, which uses single layer classification head (nn.Linear)

[CISC]

Nice, I was just looking into adding more Bert models in fact. :)

[ggerganov]

Nice, I was just looking into adding more Bert models in fact. :)

We should add ModernBERT.

[huydt84]

@CISC I fixed the linting issues.

[CISC]

Hmmm, this broke rerank-tiny (JinaBert):

Traceback (most recent call last):
  File "/home/ggml/work/llama.cpp/build-ci-release/../convert_hf_to_gguf.py", line 6446, in <module>
    main()
  File "/home/ggml/work/llama.cpp/build-ci-release/../convert_hf_to_gguf.py", line 6440, in main
    model_instance.write()
  File "/home/ggml/work/llama.cpp/build-ci-release/../convert_hf_to_gguf.py", line 403, in write
    self.prepare_tensors()
  File "/home/ggml/work/llama.cpp/build-ci-release/../convert_hf_to_gguf.py", line 277, in prepare_tensors
    for new_name, data_torch in (self.modify_tensors(data_torch, name, bid)):
  File "/home/ggml/work/llama.cpp/build-ci-release/../convert_hf_to_gguf.py", line 4755, in modify_tensors
    return super().modify_tensors(data_torch, name, bid)
  File "/home/ggml/work/llama.cpp/build-ci-release/../convert_hf_to_gguf.py", line 3755, in modify_tensors
    return [(self.map_tensor_name(name), data_torch)]
  File "/home/ggml/work/llama.cpp/build-ci-release/../convert_hf_to_gguf.py", line 236, in map_tensor_name
    raise ValueError(f"Can not map tensor {name!r}")
ValueError: Can not map tensor 'classifier.out_proj.weight'

[CISC]

@huydt84 Can you make another PR that fixes this?

We need a way to identify this type of model, it looks like num_labels might be a way?
https://huggingface.co/webis/tiny-bert-ranker/blob/main/config.json#L11-L13

IE, just hparams.get("id2label") in modify_tensors and only modify classifier if it exists.

[huydt84]

@CISC Let me check that.

[CISC]

@CISC Let me check that.

Actually, nvm, I'm adding Roberta reranker, so it's convenient to add the fix there (it has 2 labels, so requires special handling anyway).