Update README.md
Browse files
README.md
CHANGED
@@ -150,7 +150,26 @@ datasets:
|
|
150 |
|
151 |
# SPLADE-BERT-Tiny-Distil
|
152 |
|
153 |
-
This is a
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
154 |
|
155 |
## Usage
|
156 |
|
|
|
150 |
|
151 |
# SPLADE-BERT-Tiny-Distil
|
152 |
|
153 |
+
This is a SPLADE sparse retrieval models based on BERT-Tiny (4M) trained by distilling a Cross-Encoder on the MSMARCO dataset. The cross-encoder used was [ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2).
|
154 |
+
|
155 |
+
The tiny SPLADE models are `6-15x` smaller than Naver's official `splade-v3-distilbert` while having `80-85%` of it's performance on the MSMARCO benchmark. The models are small enough to be used without a GPU on a dataset of a few thousand documents.
|
156 |
+
|
157 |
+
You can download the models from the following huggingface collection.
|
158 |
+
|
159 |
+
- Models: https://huggingface.co/collections/rasyosef/splade-tiny-msmarco-687c548c0691d95babf65b70
|
160 |
+
- Distillation Dataset: https://huggingface.co/datasets/yosefw/msmarco-train-distil-v2
|
161 |
+
- Code: https://github.com/rasyosef/splade-tiny-msmarco
|
162 |
+
|
163 |
+
## Performance
|
164 |
+
|
165 |
+
The splade models were evaluated on 55 thousand queries and 8 million documents from the [MSMARCO](https://huggingface.co/datasets/microsoft/ms_marco) dataset.
|
166 |
+
|
167 |
+
||Size (# Params)|MRR@10 (MS MARCO dev)|
|
168 |
+
|:---|:----|:-------------------|
|
169 |
+
|`BM25`|-|18.6|-|-|
|
170 |
+
|`rasyosef/splade-tiny`|4.4M|30.8|
|
171 |
+
|`rasyosef/splade-mini`|11.2M|32.8|
|
172 |
+
|`naver/splade-v3-distilbert`|67.0M|38.7|
|
173 |
|
174 |
## Usage
|
175 |
|