Update README.md
Browse files
README.md
CHANGED
|
@@ -9,6 +9,25 @@ language:
|
|
| 9 |
pipeline_tag: sentence-similarity
|
| 10 |
---
|
| 11 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
## Evaluation
|
| 13 |
|
| 14 |
To assess the performance of the reranker, we will utilize the "validation" split of the [SQuAD](https://huggingface.co/datasets/rajpurkar/squad) dataset. We will select
|
|
|
|
| 9 |
pipeline_tag: sentence-similarity
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# Bloomz-3b Reranking
|
| 13 |
+
|
| 14 |
+
This reranking model is built from [cmarkea/bloomz-3b-dpo-chat](https://huggingface.co/cmarkea/bloomz-3b-dpo-chat) model and aims to gauge the correspondence between
|
| 15 |
+
a question (query) and a context. With its normalized scoring, it facilitates filtering of results derived from query/context matches at the output of a retriever.
|
| 16 |
+
Moreover, it enables the reordering of results using a modeling approach more efficient than the retriever's. However, this modeling type is not conducive to direct
|
| 17 |
+
database searching due to its high computational cost.
|
| 18 |
+
|
| 19 |
+
Developed to be language-agnostic, this model supports both French and English. Consequently, it can effectively score in a cross-language context without being
|
| 20 |
+
influenced by its behavior in a monolingual context (English or French).
|
| 21 |
+
|
| 22 |
+
## Dataset
|
| 23 |
+
The training dataset comprises the [mMARCO dataset](https://huggingface.co/datasets/unicamp-dl/mmarco), consisting of query/positive/hard negative triplets. Additionally,
|
| 24 |
+
we have included [SQuAD](https://huggingface.co/datasets/rajpurkar/squad) data from the train split, forming query/positive/hard negative triplets. To generate hard
|
| 25 |
+
negative data for SQuAD, we considered contexts from the same theme as the query but from a different set of queries. Hence, the negative observations address the same
|
| 26 |
+
themes as the queries but presumably do not contain the answer to the question.
|
| 27 |
+
|
| 28 |
+
Finally, the triplets are flattened to obtain pairs of query/context sentences with a label of 1 if query/positive and a label 0 if query/negative. In each element of the
|
| 29 |
+
pair (query/context), the language, French or English, is randomly and uniformly chosen.
|
| 30 |
+
|
| 31 |
## Evaluation
|
| 32 |
|
| 33 |
To assess the performance of the reranker, we will utilize the "validation" split of the [SQuAD](https://huggingface.co/datasets/rajpurkar/squad) dataset. We will select
|