rasyosef commited on
Commit
3677c5b
·
verified ·
1 Parent(s): de3ab2c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -1
README.md CHANGED
@@ -150,7 +150,26 @@ datasets:
150
 
151
  # SPLADE-BERT-Tiny-Distil
152
 
153
- This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [prajjwal1/bert-tiny](https://huggingface.co/prajjwal1/bert-tiny) using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
154
 
155
  ## Usage
156
 
 
150
 
151
  # SPLADE-BERT-Tiny-Distil
152
 
153
+ This is a SPLADE sparse retrieval models based on BERT-Tiny (4M) trained by distilling a Cross-Encoder on the MSMARCO dataset. The cross-encoder used was [ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2).
154
+
155
+ The tiny SPLADE models are `6-15x` smaller than Naver's official `splade-v3-distilbert` while having `80-85%` of it's performance on the MSMARCO benchmark. The models are small enough to be used without a GPU on a dataset of a few thousand documents.
156
+
157
+ You can download the models from the following huggingface collection.
158
+
159
+ - Models: https://huggingface.co/collections/rasyosef/splade-tiny-msmarco-687c548c0691d95babf65b70
160
+ - Distillation Dataset: https://huggingface.co/datasets/yosefw/msmarco-train-distil-v2
161
+ - Code: https://github.com/rasyosef/splade-tiny-msmarco
162
+
163
+ ## Performance
164
+
165
+ The splade models were evaluated on 55 thousand queries and 8 million documents from the [MSMARCO](https://huggingface.co/datasets/microsoft/ms_marco) dataset.
166
+
167
+ ||Size (# Params)|MRR@10 (MS MARCO dev)|
168
+ |:---|:----|:-------------------|
169
+ |`BM25`|-|18.6|-|-|
170
+ |`rasyosef/splade-tiny`|4.4M|30.8|
171
+ |`rasyosef/splade-mini`|11.2M|32.8|
172
+ |`naver/splade-v3-distilbert`|67.0M|38.7|
173
 
174
  ## Usage
175