REBEL - new node postprocessor

[bvarjavand]

Description

This PR introduces REBELRerank, a novel node postprocessor that implements the REBEL (RErank BEyond reLevance) methodology for multi-criteria reranking in RAG pipelines. REBEL extends traditional relevance-centric reranking by incorporating query-dependent secondary criteria through dynamic prompt generation.

This PR is still in progress.

Key features:

• Meta-prompting strategy to generate query-specific reranking instructions
• Integration of multiple scoring criteria beyond just relevance
• Support for weighted composite scoring that considers factors like authoritativeness, diversity, and recency
• Fully prompt-based implementation requiring no model fine-tuning

The PR includes:

• Implementation of REBELRerank as a node postprocessor
• Demonstration notebook showing usage and benefits
• Documentation and tests (TODO)

New Package?

• No

Version Bump?

• Yes

Type of Change

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

• I added new unit tests to cover this change (TODO)

Unit tests will cover:

• Meta prompt generation
• Reranking prompt generation

Suggested Checklist (TODO):

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added Google Colab support for the newly added notebooks
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I linted

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[bvarjavand]

The code to generate the figures in the paper lives here: https://github.com/levinwil/REBEL/tree/main
We will submit to arxiv after some Cohere comparisons locally.

[logan-markewich]

@bvarjavand is this still in progress? I can help clean up the CICD, but don't want to step on any toes before I do that

[bvarjavand]

@logan-markewich apologies for the delay. I've added a unit test ensuring it works, and linted the files. This along with the notebook for the docs should be sufficient. Please let me know what else you may need.

[bvarjavand]

resolved conflicts, please review :)

[bvarjavand]

ping for visibility

[bvarjavand]

@logan-markewich or anyone who would review this, happy to push this over the finish line

[github-actions]

This PR is stale because it has been open 50 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[github-actions]

This PR was closed because it has been stalled for 10 days with no activity.