Roadmap

[bpiwowar]

This tracks down all the changes planned in xpmir

Enhancements

• Support PyTerrier #3
• Tests for paper reproductions (experimental plans) #13
• Support ranxhub #30

New models

• Implement ColBERTv2 #7
• Implement DuoBERT #6
• ANCE #10
• Implement Neural Corpus Indexer #17

New pipelines

• Hierarchic clustering of documents (used in NCI) #18
• Generation of pseudo-queries using T5 (used in NCI) #19

Refactor

• Move ir-datasets support to datamaestro_text #16
• Use DataLoader / DataSets to prepare multi-GPU support
• Use accelerate for multi-GPU support

Breaking changes

• Explicit query and document text fields #15
• Fully switch to ir-datasets for data structures #11
• Move Learner to a new learning module #12
• Accessing pre-trained sub-modules #14

Final touches

• Retrain the different HF hub models so they are compatible with the stable API #20
• Document all configurations and tasks #21