Error occurred in hard negative examples mining #1339
Description
已安装faiss-gpu-cu11，遇到以下报错何解？感谢~~
机器：4*v100 16G
运行命令：
python hn_mine.py
--input_file toy_finetune_data.jsonl
--output_file toy_finetune_data_minedHN.jsonl
--range_for_sampling 5-8
--negative_number 2
--use_gpu_for_searching
--embedder_name_or_path ../../BAAI/bge-m3
inferencing embedding for corpus (number=80)--------------
initial target device: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:13<00:00, 3.31s/it]
Chunks: 0%| | 0/4 [00:00<?, ?it/s]
Chunks: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:13<00:00, 3.31s/it]
inferencing embedding for queries (number=10)--------------
Chunks: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 67.19it/s]
create index and search------------------
Traceback (most recent call last):
File "/mnt/myspace/FlagEmbedding/scripts/hn_mine.py", line 233, in
main(data_args, model_args)
File "/mnt/myspace/FlagEmbedding/scripts/hn_mine.py", line 214, in main
find_knn_neg(
File "/mnt/myspace/FlagEmbedding/scripts/hn_mine.py", line 164, in find_knn_neg
index = create_index(p_vecs, use_gpu=use_gpu)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/myspace/FlagEmbedding/scripts/hn_mine.py", line 96, in create_index
index = faiss.IndexFlatIP(len(embeddings[0]))
~~~~~~~~~~^^^
KeyError: 0