Implementing re-ranking algorithm as post-RAG

[NISH1001]

Currently the approach to RAG in our multi retrieval engine is:

• Create retrievers/indexers (Sinequa, VectorStore, SQL Agent)
• Use larch.search.engines.MultiRetrieverSearchEngine that basically does LLM-based prompting combining all the responses from each retriever

However, a natural problem here is, even if we get K chunks/evidence from each retriever, we can't know for sure the actual flat list of evidence we can combine from all these sources. And sometimes one retriever has the same evidence at top-1 and maybe at top-3, or something.

So, we need a mechanism to re-rank once everything is processed. This could potentially remove unnecessary chunks to surface at the top when users see the evidence.

We could also use the re-ranking right before generating final response, after getting chunks from each retriever.

Reciprocal Rank Fusion

One way to handle this is to use RRF (Reciprocal Rank Fusion):
https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf

Re-ranker model

Something like ML-based llm-blender could be used to do re-ranking.