tomaarsen HF Staff commited on
Commit
3dafdaf
·
verified ·
1 Parent(s): 5b0e5b3

Add new SparseEncoder model

Browse files
1_SpladePooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "pooling_strategy": "max",
3
+ "activation_function": "relu",
4
+ "word_embedding_dimension": 30522
5
+ }
README.md ADDED
@@ -0,0 +1,1731 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ tags:
6
+ - sentence-transformers
7
+ - sparse-encoder
8
+ - sparse
9
+ - splade
10
+ - generated_from_trainer
11
+ - dataset_size:99000
12
+ - loss:SpladeLoss
13
+ - loss:SparseMultipleNegativesRankingLoss
14
+ - loss:FlopsLoss
15
+ base_model: distilbert/distilbert-base-uncased
16
+ widget:
17
+ - text: 'The term emergent literacy signals a belief that, in a literate society,
18
+ young children even one and two year olds, are in the process of becoming literate”.
19
+ ... Gray (1956:21) notes: Functional literacy is used for the training of adults
20
+ to ''meet independently the reading and writing demands placed on them''.'
21
+ - text: Rey is seemingly confirmed as being The Chosen One per a quote by a Lucasfilm
22
+ production designer who worked on The Rise of Skywalker.
23
+ - text: are union gun safes fireproof?
24
+ - text: Fruit is an essential part of a healthy diet — and may aid weight loss. Most
25
+ fruits are low in calories while high in nutrients and fiber, which can boost
26
+ your fullness. Keep in mind that it's best to eat fruits whole rather than juiced.
27
+ What's more, simply eating fruit is not the key to weight loss.
28
+ - text: Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate
29
+ or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks
30
+ for chronic sinusitis.
31
+ datasets:
32
+ - sentence-transformers/gooaq
33
+ pipeline_tag: feature-extraction
34
+ library_name: sentence-transformers
35
+ metrics:
36
+ - dot_accuracy@1
37
+ - dot_accuracy@3
38
+ - dot_accuracy@5
39
+ - dot_accuracy@10
40
+ - dot_precision@1
41
+ - dot_precision@3
42
+ - dot_precision@5
43
+ - dot_precision@10
44
+ - dot_recall@1
45
+ - dot_recall@3
46
+ - dot_recall@5
47
+ - dot_recall@10
48
+ - dot_ndcg@10
49
+ - dot_mrr@10
50
+ - dot_map@100
51
+ - query_active_dims
52
+ - query_sparsity_ratio
53
+ - corpus_active_dims
54
+ - corpus_sparsity_ratio
55
+ co2_eq_emissions:
56
+ emissions: 13.144676625187973
57
+ energy_consumed: 0.03381684844736578
58
+ source: codecarbon
59
+ training_type: fine-tuning
60
+ on_cloud: false
61
+ cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
62
+ ram_total_size: 31.777088165283203
63
+ hours_used: 0.145
64
+ hardware_used: 1 x NVIDIA GeForce RTX 3090
65
+ model-index:
66
+ - name: splade-distilbert-base-uncased trained on GooAQ
67
+ results:
68
+ - task:
69
+ type: sparse-information-retrieval
70
+ name: Sparse Information Retrieval
71
+ dataset:
72
+ name: NanoMSMARCO
73
+ type: NanoMSMARCO
74
+ metrics:
75
+ - type: dot_accuracy@1
76
+ value: 0.22
77
+ name: Dot Accuracy@1
78
+ - type: dot_accuracy@3
79
+ value: 0.4
80
+ name: Dot Accuracy@3
81
+ - type: dot_accuracy@5
82
+ value: 0.5
83
+ name: Dot Accuracy@5
84
+ - type: dot_accuracy@10
85
+ value: 0.7
86
+ name: Dot Accuracy@10
87
+ - type: dot_precision@1
88
+ value: 0.22
89
+ name: Dot Precision@1
90
+ - type: dot_precision@3
91
+ value: 0.13333333333333333
92
+ name: Dot Precision@3
93
+ - type: dot_precision@5
94
+ value: 0.1
95
+ name: Dot Precision@5
96
+ - type: dot_precision@10
97
+ value: 0.07
98
+ name: Dot Precision@10
99
+ - type: dot_recall@1
100
+ value: 0.22
101
+ name: Dot Recall@1
102
+ - type: dot_recall@3
103
+ value: 0.4
104
+ name: Dot Recall@3
105
+ - type: dot_recall@5
106
+ value: 0.5
107
+ name: Dot Recall@5
108
+ - type: dot_recall@10
109
+ value: 0.7
110
+ name: Dot Recall@10
111
+ - type: dot_ndcg@10
112
+ value: 0.43322728177988873
113
+ name: Dot Ndcg@10
114
+ - type: dot_mrr@10
115
+ value: 0.35121428571428576
116
+ name: Dot Mrr@10
117
+ - type: dot_map@100
118
+ value: 0.36254438939466105
119
+ name: Dot Map@100
120
+ - type: query_active_dims
121
+ value: 114.83999633789062
122
+ name: Query Active Dims
123
+ - type: query_sparsity_ratio
124
+ value: 0.9962374681758112
125
+ name: Query Sparsity Ratio
126
+ - type: corpus_active_dims
127
+ value: 504.9510192871094
128
+ name: Corpus Active Dims
129
+ - type: corpus_sparsity_ratio
130
+ value: 0.9834561621359311
131
+ name: Corpus Sparsity Ratio
132
+ - type: dot_accuracy@1
133
+ value: 0.22
134
+ name: Dot Accuracy@1
135
+ - type: dot_accuracy@3
136
+ value: 0.4
137
+ name: Dot Accuracy@3
138
+ - type: dot_accuracy@5
139
+ value: 0.5
140
+ name: Dot Accuracy@5
141
+ - type: dot_accuracy@10
142
+ value: 0.7
143
+ name: Dot Accuracy@10
144
+ - type: dot_precision@1
145
+ value: 0.22
146
+ name: Dot Precision@1
147
+ - type: dot_precision@3
148
+ value: 0.13333333333333333
149
+ name: Dot Precision@3
150
+ - type: dot_precision@5
151
+ value: 0.1
152
+ name: Dot Precision@5
153
+ - type: dot_precision@10
154
+ value: 0.07
155
+ name: Dot Precision@10
156
+ - type: dot_recall@1
157
+ value: 0.22
158
+ name: Dot Recall@1
159
+ - type: dot_recall@3
160
+ value: 0.4
161
+ name: Dot Recall@3
162
+ - type: dot_recall@5
163
+ value: 0.5
164
+ name: Dot Recall@5
165
+ - type: dot_recall@10
166
+ value: 0.7
167
+ name: Dot Recall@10
168
+ - type: dot_ndcg@10
169
+ value: 0.43322728177988873
170
+ name: Dot Ndcg@10
171
+ - type: dot_mrr@10
172
+ value: 0.35121428571428576
173
+ name: Dot Mrr@10
174
+ - type: dot_map@100
175
+ value: 0.36254438939466105
176
+ name: Dot Map@100
177
+ - type: query_active_dims
178
+ value: 114.83999633789062
179
+ name: Query Active Dims
180
+ - type: query_sparsity_ratio
181
+ value: 0.9962374681758112
182
+ name: Query Sparsity Ratio
183
+ - type: corpus_active_dims
184
+ value: 504.9510192871094
185
+ name: Corpus Active Dims
186
+ - type: corpus_sparsity_ratio
187
+ value: 0.9834561621359311
188
+ name: Corpus Sparsity Ratio
189
+ - task:
190
+ type: sparse-information-retrieval
191
+ name: Sparse Information Retrieval
192
+ dataset:
193
+ name: NanoNFCorpus
194
+ type: NanoNFCorpus
195
+ metrics:
196
+ - type: dot_accuracy@1
197
+ value: 0.28
198
+ name: Dot Accuracy@1
199
+ - type: dot_accuracy@3
200
+ value: 0.42
201
+ name: Dot Accuracy@3
202
+ - type: dot_accuracy@5
203
+ value: 0.44
204
+ name: Dot Accuracy@5
205
+ - type: dot_accuracy@10
206
+ value: 0.48
207
+ name: Dot Accuracy@10
208
+ - type: dot_precision@1
209
+ value: 0.28
210
+ name: Dot Precision@1
211
+ - type: dot_precision@3
212
+ value: 0.2533333333333333
213
+ name: Dot Precision@3
214
+ - type: dot_precision@5
215
+ value: 0.20800000000000002
216
+ name: Dot Precision@5
217
+ - type: dot_precision@10
218
+ value: 0.172
219
+ name: Dot Precision@10
220
+ - type: dot_recall@1
221
+ value: 0.01025265789874976
222
+ name: Dot Recall@1
223
+ - type: dot_recall@3
224
+ value: 0.024326098686792398
225
+ name: Dot Recall@3
226
+ - type: dot_recall@5
227
+ value: 0.03315745551680213
228
+ name: Dot Recall@5
229
+ - type: dot_recall@10
230
+ value: 0.058486915473213524
231
+ name: Dot Recall@10
232
+ - type: dot_ndcg@10
233
+ value: 0.19719700869611326
234
+ name: Dot Ndcg@10
235
+ - type: dot_mrr@10
236
+ value: 0.35035714285714287
237
+ name: Dot Mrr@10
238
+ - type: dot_map@100
239
+ value: 0.06408607089134896
240
+ name: Dot Map@100
241
+ - type: query_active_dims
242
+ value: 185.0
243
+ name: Query Active Dims
244
+ - type: query_sparsity_ratio
245
+ value: 0.9939387982438896
246
+ name: Query Sparsity Ratio
247
+ - type: corpus_active_dims
248
+ value: 1286.7938232421875
249
+ name: Corpus Active Dims
250
+ - type: corpus_sparsity_ratio
251
+ value: 0.9578404487503379
252
+ name: Corpus Sparsity Ratio
253
+ - type: dot_accuracy@1
254
+ value: 0.28
255
+ name: Dot Accuracy@1
256
+ - type: dot_accuracy@3
257
+ value: 0.42
258
+ name: Dot Accuracy@3
259
+ - type: dot_accuracy@5
260
+ value: 0.44
261
+ name: Dot Accuracy@5
262
+ - type: dot_accuracy@10
263
+ value: 0.48
264
+ name: Dot Accuracy@10
265
+ - type: dot_precision@1
266
+ value: 0.28
267
+ name: Dot Precision@1
268
+ - type: dot_precision@3
269
+ value: 0.2533333333333333
270
+ name: Dot Precision@3
271
+ - type: dot_precision@5
272
+ value: 0.20800000000000002
273
+ name: Dot Precision@5
274
+ - type: dot_precision@10
275
+ value: 0.172
276
+ name: Dot Precision@10
277
+ - type: dot_recall@1
278
+ value: 0.01025265789874976
279
+ name: Dot Recall@1
280
+ - type: dot_recall@3
281
+ value: 0.024326098686792398
282
+ name: Dot Recall@3
283
+ - type: dot_recall@5
284
+ value: 0.03315745551680213
285
+ name: Dot Recall@5
286
+ - type: dot_recall@10
287
+ value: 0.058486915473213524
288
+ name: Dot Recall@10
289
+ - type: dot_ndcg@10
290
+ value: 0.19719700869611326
291
+ name: Dot Ndcg@10
292
+ - type: dot_mrr@10
293
+ value: 0.35035714285714287
294
+ name: Dot Mrr@10
295
+ - type: dot_map@100
296
+ value: 0.06408607089134896
297
+ name: Dot Map@100
298
+ - type: query_active_dims
299
+ value: 185.0
300
+ name: Query Active Dims
301
+ - type: query_sparsity_ratio
302
+ value: 0.9939387982438896
303
+ name: Query Sparsity Ratio
304
+ - type: corpus_active_dims
305
+ value: 1286.7938232421875
306
+ name: Corpus Active Dims
307
+ - type: corpus_sparsity_ratio
308
+ value: 0.9578404487503379
309
+ name: Corpus Sparsity Ratio
310
+ - task:
311
+ type: sparse-information-retrieval
312
+ name: Sparse Information Retrieval
313
+ dataset:
314
+ name: NanoNQ
315
+ type: NanoNQ
316
+ metrics:
317
+ - type: dot_accuracy@1
318
+ value: 0.22
319
+ name: Dot Accuracy@1
320
+ - type: dot_accuracy@3
321
+ value: 0.36
322
+ name: Dot Accuracy@3
323
+ - type: dot_accuracy@5
324
+ value: 0.5
325
+ name: Dot Accuracy@5
326
+ - type: dot_accuracy@10
327
+ value: 0.56
328
+ name: Dot Accuracy@10
329
+ - type: dot_precision@1
330
+ value: 0.22
331
+ name: Dot Precision@1
332
+ - type: dot_precision@3
333
+ value: 0.12
334
+ name: Dot Precision@3
335
+ - type: dot_precision@5
336
+ value: 0.10000000000000002
337
+ name: Dot Precision@5
338
+ - type: dot_precision@10
339
+ value: 0.05600000000000001
340
+ name: Dot Precision@10
341
+ - type: dot_recall@1
342
+ value: 0.2
343
+ name: Dot Recall@1
344
+ - type: dot_recall@3
345
+ value: 0.33
346
+ name: Dot Recall@3
347
+ - type: dot_recall@5
348
+ value: 0.46
349
+ name: Dot Recall@5
350
+ - type: dot_recall@10
351
+ value: 0.52
352
+ name: Dot Recall@10
353
+ - type: dot_ndcg@10
354
+ value: 0.3556861493087894
355
+ name: Dot Ndcg@10
356
+ - type: dot_mrr@10
357
+ value: 0.322
358
+ name: Dot Mrr@10
359
+ - type: dot_map@100
360
+ value: 0.31376550096906575
361
+ name: Dot Map@100
362
+ - type: query_active_dims
363
+ value: 98.22000122070312
364
+ name: Query Active Dims
365
+ - type: query_sparsity_ratio
366
+ value: 0.9967819932763022
367
+ name: Query Sparsity Ratio
368
+ - type: corpus_active_dims
369
+ value: 841.8667602539062
370
+ name: Corpus Active Dims
371
+ - type: corpus_sparsity_ratio
372
+ value: 0.9724177065639898
373
+ name: Corpus Sparsity Ratio
374
+ - type: dot_accuracy@1
375
+ value: 0.22
376
+ name: Dot Accuracy@1
377
+ - type: dot_accuracy@3
378
+ value: 0.36
379
+ name: Dot Accuracy@3
380
+ - type: dot_accuracy@5
381
+ value: 0.5
382
+ name: Dot Accuracy@5
383
+ - type: dot_accuracy@10
384
+ value: 0.56
385
+ name: Dot Accuracy@10
386
+ - type: dot_precision@1
387
+ value: 0.22
388
+ name: Dot Precision@1
389
+ - type: dot_precision@3
390
+ value: 0.12
391
+ name: Dot Precision@3
392
+ - type: dot_precision@5
393
+ value: 0.10000000000000002
394
+ name: Dot Precision@5
395
+ - type: dot_precision@10
396
+ value: 0.05600000000000001
397
+ name: Dot Precision@10
398
+ - type: dot_recall@1
399
+ value: 0.2
400
+ name: Dot Recall@1
401
+ - type: dot_recall@3
402
+ value: 0.33
403
+ name: Dot Recall@3
404
+ - type: dot_recall@5
405
+ value: 0.46
406
+ name: Dot Recall@5
407
+ - type: dot_recall@10
408
+ value: 0.52
409
+ name: Dot Recall@10
410
+ - type: dot_ndcg@10
411
+ value: 0.3556861493087894
412
+ name: Dot Ndcg@10
413
+ - type: dot_mrr@10
414
+ value: 0.322
415
+ name: Dot Mrr@10
416
+ - type: dot_map@100
417
+ value: 0.31376550096906575
418
+ name: Dot Map@100
419
+ - type: query_active_dims
420
+ value: 98.22000122070312
421
+ name: Query Active Dims
422
+ - type: query_sparsity_ratio
423
+ value: 0.9967819932763022
424
+ name: Query Sparsity Ratio
425
+ - type: corpus_active_dims
426
+ value: 841.8667602539062
427
+ name: Corpus Active Dims
428
+ - type: corpus_sparsity_ratio
429
+ value: 0.9724177065639898
430
+ name: Corpus Sparsity Ratio
431
+ - task:
432
+ type: sparse-nano-beir
433
+ name: Sparse Nano BEIR
434
+ dataset:
435
+ name: NanoBEIR mean
436
+ type: NanoBEIR_mean
437
+ metrics:
438
+ - type: dot_accuracy@1
439
+ value: 0.24
440
+ name: Dot Accuracy@1
441
+ - type: dot_accuracy@3
442
+ value: 0.39333333333333337
443
+ name: Dot Accuracy@3
444
+ - type: dot_accuracy@5
445
+ value: 0.48
446
+ name: Dot Accuracy@5
447
+ - type: dot_accuracy@10
448
+ value: 0.58
449
+ name: Dot Accuracy@10
450
+ - type: dot_precision@1
451
+ value: 0.24
452
+ name: Dot Precision@1
453
+ - type: dot_precision@3
454
+ value: 0.16888888888888887
455
+ name: Dot Precision@3
456
+ - type: dot_precision@5
457
+ value: 0.13600000000000004
458
+ name: Dot Precision@5
459
+ - type: dot_precision@10
460
+ value: 0.09933333333333333
461
+ name: Dot Precision@10
462
+ - type: dot_recall@1
463
+ value: 0.14341755263291658
464
+ name: Dot Recall@1
465
+ - type: dot_recall@3
466
+ value: 0.25144203289559747
467
+ name: Dot Recall@3
468
+ - type: dot_recall@5
469
+ value: 0.3310524851722674
470
+ name: Dot Recall@5
471
+ - type: dot_recall@10
472
+ value: 0.42616230515773784
473
+ name: Dot Recall@10
474
+ - type: dot_ndcg@10
475
+ value: 0.3287034799282638
476
+ name: Dot Ndcg@10
477
+ - type: dot_mrr@10
478
+ value: 0.3411904761904762
479
+ name: Dot Mrr@10
480
+ - type: dot_map@100
481
+ value: 0.24679865375169194
482
+ name: Dot Map@100
483
+ - type: query_active_dims
484
+ value: 132.6866658528646
485
+ name: Query Active Dims
486
+ - type: query_sparsity_ratio
487
+ value: 0.9956527532320011
488
+ name: Query Sparsity Ratio
489
+ - type: corpus_active_dims
490
+ value: 812.3067522198979
491
+ name: Corpus Active Dims
492
+ - type: corpus_sparsity_ratio
493
+ value: 0.9733861885780781
494
+ name: Corpus Sparsity Ratio
495
+ - type: dot_accuracy@1
496
+ value: 0.3254945054945055
497
+ name: Dot Accuracy@1
498
+ - type: dot_accuracy@3
499
+ value: 0.4843328100470958
500
+ name: Dot Accuracy@3
501
+ - type: dot_accuracy@5
502
+ value: 0.5676295133437991
503
+ name: Dot Accuracy@5
504
+ - type: dot_accuracy@10
505
+ value: 0.6615384615384615
506
+ name: Dot Accuracy@10
507
+ - type: dot_precision@1
508
+ value: 0.3254945054945055
509
+ name: Dot Precision@1
510
+ - type: dot_precision@3
511
+ value: 0.2203453689167975
512
+ name: Dot Precision@3
513
+ - type: dot_precision@5
514
+ value: 0.1832904238618524
515
+ name: Dot Precision@5
516
+ - type: dot_precision@10
517
+ value: 0.13404081632653062
518
+ name: Dot Precision@10
519
+ - type: dot_recall@1
520
+ value: 0.17156366311931473
521
+ name: Dot Recall@1
522
+ - type: dot_recall@3
523
+ value: 0.27243997398612047
524
+ name: Dot Recall@3
525
+ - type: dot_recall@5
526
+ value: 0.3368199222866662
527
+ name: Dot Recall@5
528
+ - type: dot_recall@10
529
+ value: 0.4238029847392705
530
+ name: Dot Recall@10
531
+ - type: dot_ndcg@10
532
+ value: 0.3726337418448364
533
+ name: Dot Ndcg@10
534
+ - type: dot_mrr@10
535
+ value: 0.4264663726296379
536
+ name: Dot Mrr@10
537
+ - type: dot_map@100
538
+ value: 0.2989418038202097
539
+ name: Dot Map@100
540
+ - type: query_active_dims
541
+ value: 234.31433094300914
542
+ name: Query Active Dims
543
+ - type: query_sparsity_ratio
544
+ value: 0.9923231003557103
545
+ name: Query Sparsity Ratio
546
+ - type: corpus_active_dims
547
+ value: 808.1458433081926
548
+ name: Corpus Active Dims
549
+ - type: corpus_sparsity_ratio
550
+ value: 0.9735225134883626
551
+ name: Corpus Sparsity Ratio
552
+ - task:
553
+ type: sparse-information-retrieval
554
+ name: Sparse Information Retrieval
555
+ dataset:
556
+ name: NanoClimateFEVER
557
+ type: NanoClimateFEVER
558
+ metrics:
559
+ - type: dot_accuracy@1
560
+ value: 0.22
561
+ name: Dot Accuracy@1
562
+ - type: dot_accuracy@3
563
+ value: 0.28
564
+ name: Dot Accuracy@3
565
+ - type: dot_accuracy@5
566
+ value: 0.36
567
+ name: Dot Accuracy@5
568
+ - type: dot_accuracy@10
569
+ value: 0.46
570
+ name: Dot Accuracy@10
571
+ - type: dot_precision@1
572
+ value: 0.22
573
+ name: Dot Precision@1
574
+ - type: dot_precision@3
575
+ value: 0.11333333333333333
576
+ name: Dot Precision@3
577
+ - type: dot_precision@5
578
+ value: 0.084
579
+ name: Dot Precision@5
580
+ - type: dot_precision@10
581
+ value: 0.05800000000000001
582
+ name: Dot Precision@10
583
+ - type: dot_recall@1
584
+ value: 0.09166666666666666
585
+ name: Dot Recall@1
586
+ - type: dot_recall@3
587
+ value: 0.15333333333333332
588
+ name: Dot Recall@3
589
+ - type: dot_recall@5
590
+ value: 0.17666666666666664
591
+ name: Dot Recall@5
592
+ - type: dot_recall@10
593
+ value: 0.22466666666666665
594
+ name: Dot Recall@10
595
+ - type: dot_ndcg@10
596
+ value: 0.19429559758090853
597
+ name: Dot Ndcg@10
598
+ - type: dot_mrr@10
599
+ value: 0.27672222222222226
600
+ name: Dot Mrr@10
601
+ - type: dot_map@100
602
+ value: 0.15485373044420248
603
+ name: Dot Map@100
604
+ - type: query_active_dims
605
+ value: 259.8599853515625
606
+ name: Query Active Dims
607
+ - type: query_sparsity_ratio
608
+ value: 0.9914861416240233
609
+ name: Query Sparsity Ratio
610
+ - type: corpus_active_dims
611
+ value: 1094.6026611328125
612
+ name: Corpus Active Dims
613
+ - type: corpus_sparsity_ratio
614
+ value: 0.9641372563681013
615
+ name: Corpus Sparsity Ratio
616
+ - task:
617
+ type: sparse-information-retrieval
618
+ name: Sparse Information Retrieval
619
+ dataset:
620
+ name: NanoDBPedia
621
+ type: NanoDBPedia
622
+ metrics:
623
+ - type: dot_accuracy@1
624
+ value: 0.52
625
+ name: Dot Accuracy@1
626
+ - type: dot_accuracy@3
627
+ value: 0.74
628
+ name: Dot Accuracy@3
629
+ - type: dot_accuracy@5
630
+ value: 0.84
631
+ name: Dot Accuracy@5
632
+ - type: dot_accuracy@10
633
+ value: 0.88
634
+ name: Dot Accuracy@10
635
+ - type: dot_precision@1
636
+ value: 0.52
637
+ name: Dot Precision@1
638
+ - type: dot_precision@3
639
+ value: 0.4599999999999999
640
+ name: Dot Precision@3
641
+ - type: dot_precision@5
642
+ value: 0.45199999999999996
643
+ name: Dot Precision@5
644
+ - type: dot_precision@10
645
+ value: 0.384
646
+ name: Dot Precision@10
647
+ - type: dot_recall@1
648
+ value: 0.04966217676438495
649
+ name: Dot Recall@1
650
+ - type: dot_recall@3
651
+ value: 0.10354828293616407
652
+ name: Dot Recall@3
653
+ - type: dot_recall@5
654
+ value: 0.16425525763608173
655
+ name: Dot Recall@5
656
+ - type: dot_recall@10
657
+ value: 0.2406829559845734
658
+ name: Dot Recall@10
659
+ - type: dot_ndcg@10
660
+ value: 0.456594069464261
661
+ name: Dot Ndcg@10
662
+ - type: dot_mrr@10
663
+ value: 0.6436666666666666
664
+ name: Dot Mrr@10
665
+ - type: dot_map@100
666
+ value: 0.3020935356938311
667
+ name: Dot Map@100
668
+ - type: query_active_dims
669
+ value: 191.25999450683594
670
+ name: Query Active Dims
671
+ - type: query_sparsity_ratio
672
+ value: 0.9937337004617379
673
+ name: Query Sparsity Ratio
674
+ - type: corpus_active_dims
675
+ value: 809.2098999023438
676
+ name: Corpus Active Dims
677
+ - type: corpus_sparsity_ratio
678
+ value: 0.9734876515332435
679
+ name: Corpus Sparsity Ratio
680
+ - task:
681
+ type: sparse-information-retrieval
682
+ name: Sparse Information Retrieval
683
+ dataset:
684
+ name: NanoFEVER
685
+ type: NanoFEVER
686
+ metrics:
687
+ - type: dot_accuracy@1
688
+ value: 0.58
689
+ name: Dot Accuracy@1
690
+ - type: dot_accuracy@3
691
+ value: 0.66
692
+ name: Dot Accuracy@3
693
+ - type: dot_accuracy@5
694
+ value: 0.72
695
+ name: Dot Accuracy@5
696
+ - type: dot_accuracy@10
697
+ value: 0.88
698
+ name: Dot Accuracy@10
699
+ - type: dot_precision@1
700
+ value: 0.58
701
+ name: Dot Precision@1
702
+ - type: dot_precision@3
703
+ value: 0.22
704
+ name: Dot Precision@3
705
+ - type: dot_precision@5
706
+ value: 0.14800000000000002
707
+ name: Dot Precision@5
708
+ - type: dot_precision@10
709
+ value: 0.08999999999999998
710
+ name: Dot Precision@10
711
+ - type: dot_recall@1
712
+ value: 0.56
713
+ name: Dot Recall@1
714
+ - type: dot_recall@3
715
+ value: 0.63
716
+ name: Dot Recall@3
717
+ - type: dot_recall@5
718
+ value: 0.7
719
+ name: Dot Recall@5
720
+ - type: dot_recall@10
721
+ value: 0.8466666666666666
722
+ name: Dot Recall@10
723
+ - type: dot_ndcg@10
724
+ value: 0.681545812563628
725
+ name: Dot Ndcg@10
726
+ - type: dot_mrr@10
727
+ value: 0.6464126984126983
728
+ name: Dot Mrr@10
729
+ - type: dot_map@100
730
+ value: 0.6296549550854825
731
+ name: Dot Map@100
732
+ - type: query_active_dims
733
+ value: 249.5399932861328
734
+ name: Query Active Dims
735
+ - type: query_sparsity_ratio
736
+ value: 0.9918242581322937
737
+ name: Query Sparsity Ratio
738
+ - type: corpus_active_dims
739
+ value: 1358.960205078125
740
+ name: Corpus Active Dims
741
+ - type: corpus_sparsity_ratio
742
+ value: 0.9554760433432237
743
+ name: Corpus Sparsity Ratio
744
+ - task:
745
+ type: sparse-information-retrieval
746
+ name: Sparse Information Retrieval
747
+ dataset:
748
+ name: NanoFiQA2018
749
+ type: NanoFiQA2018
750
+ metrics:
751
+ - type: dot_accuracy@1
752
+ value: 0.14
753
+ name: Dot Accuracy@1
754
+ - type: dot_accuracy@3
755
+ value: 0.26
756
+ name: Dot Accuracy@3
757
+ - type: dot_accuracy@5
758
+ value: 0.36
759
+ name: Dot Accuracy@5
760
+ - type: dot_accuracy@10
761
+ value: 0.46
762
+ name: Dot Accuracy@10
763
+ - type: dot_precision@1
764
+ value: 0.14
765
+ name: Dot Precision@1
766
+ - type: dot_precision@3
767
+ value: 0.10666666666666666
768
+ name: Dot Precision@3
769
+ - type: dot_precision@5
770
+ value: 0.10400000000000002
771
+ name: Dot Precision@5
772
+ - type: dot_precision@10
773
+ value: 0.068
774
+ name: Dot Precision@10
775
+ - type: dot_recall@1
776
+ value: 0.07933333333333334
777
+ name: Dot Recall@1
778
+ - type: dot_recall@3
779
+ value: 0.157
780
+ name: Dot Recall@3
781
+ - type: dot_recall@5
782
+ value: 0.2571666666666667
783
+ name: Dot Recall@5
784
+ - type: dot_recall@10
785
+ value: 0.30074603174603176
786
+ name: Dot Recall@10
787
+ - type: dot_ndcg@10
788
+ value: 0.21720208465433088
789
+ name: Dot Ndcg@10
790
+ - type: dot_mrr@10
791
+ value: 0.23057936507936508
792
+ name: Dot Mrr@10
793
+ - type: dot_map@100
794
+ value: 0.17181110132538066
795
+ name: Dot Map@100
796
+ - type: query_active_dims
797
+ value: 87.4000015258789
798
+ name: Query Active Dims
799
+ - type: query_sparsity_ratio
800
+ value: 0.9971364916609043
801
+ name: Query Sparsity Ratio
802
+ - type: corpus_active_dims
803
+ value: 517.6328735351562
804
+ name: Corpus Active Dims
805
+ - type: corpus_sparsity_ratio
806
+ value: 0.9830406633400447
807
+ name: Corpus Sparsity Ratio
808
+ - task:
809
+ type: sparse-information-retrieval
810
+ name: Sparse Information Retrieval
811
+ dataset:
812
+ name: NanoHotpotQA
813
+ type: NanoHotpotQA
814
+ metrics:
815
+ - type: dot_accuracy@1
816
+ value: 0.48
817
+ name: Dot Accuracy@1
818
+ - type: dot_accuracy@3
819
+ value: 0.64
820
+ name: Dot Accuracy@3
821
+ - type: dot_accuracy@5
822
+ value: 0.78
823
+ name: Dot Accuracy@5
824
+ - type: dot_accuracy@10
825
+ value: 0.84
826
+ name: Dot Accuracy@10
827
+ - type: dot_precision@1
828
+ value: 0.48
829
+ name: Dot Precision@1
830
+ - type: dot_precision@3
831
+ value: 0.26666666666666666
832
+ name: Dot Precision@3
833
+ - type: dot_precision@5
834
+ value: 0.204
835
+ name: Dot Precision@5
836
+ - type: dot_precision@10
837
+ value: 0.114
838
+ name: Dot Precision@10
839
+ - type: dot_recall@1
840
+ value: 0.24
841
+ name: Dot Recall@1
842
+ - type: dot_recall@3
843
+ value: 0.4
844
+ name: Dot Recall@3
845
+ - type: dot_recall@5
846
+ value: 0.51
847
+ name: Dot Recall@5
848
+ - type: dot_recall@10
849
+ value: 0.57
850
+ name: Dot Recall@10
851
+ - type: dot_ndcg@10
852
+ value: 0.489382062974203
853
+ name: Dot Ndcg@10
854
+ - type: dot_mrr@10
855
+ value: 0.5975555555555556
856
+ name: Dot Mrr@10
857
+ - type: dot_map@100
858
+ value: 0.41273857719946977
859
+ name: Dot Map@100
860
+ - type: query_active_dims
861
+ value: 151.22000122070312
862
+ name: Query Active Dims
863
+ - type: query_sparsity_ratio
864
+ value: 0.9950455408813085
865
+ name: Query Sparsity Ratio
866
+ - type: corpus_active_dims
867
+ value: 904.4683837890625
868
+ name: Corpus Active Dims
869
+ - type: corpus_sparsity_ratio
870
+ value: 0.9703666737504403
871
+ name: Corpus Sparsity Ratio
872
+ - task:
873
+ type: sparse-information-retrieval
874
+ name: Sparse Information Retrieval
875
+ dataset:
876
+ name: NanoQuoraRetrieval
877
+ type: NanoQuoraRetrieval
878
+ metrics:
879
+ - type: dot_accuracy@1
880
+ value: 0.34
881
+ name: Dot Accuracy@1
882
+ - type: dot_accuracy@3
883
+ value: 0.48
884
+ name: Dot Accuracy@3
885
+ - type: dot_accuracy@5
886
+ value: 0.58
887
+ name: Dot Accuracy@5
888
+ - type: dot_accuracy@10
889
+ value: 0.74
890
+ name: Dot Accuracy@10
891
+ - type: dot_precision@1
892
+ value: 0.34
893
+ name: Dot Precision@1
894
+ - type: dot_precision@3
895
+ value: 0.16
896
+ name: Dot Precision@3
897
+ - type: dot_precision@5
898
+ value: 0.12
899
+ name: Dot Precision@5
900
+ - type: dot_precision@10
901
+ value: 0.07800000000000001
902
+ name: Dot Precision@10
903
+ - type: dot_recall@1
904
+ value: 0.32666666666666666
905
+ name: Dot Recall@1
906
+ - type: dot_recall@3
907
+ value: 0.4466666666666667
908
+ name: Dot Recall@3
909
+ - type: dot_recall@5
910
+ value: 0.5406666666666666
911
+ name: Dot Recall@5
912
+ - type: dot_recall@10
913
+ value: 0.7106666666666667
914
+ name: Dot Recall@10
915
+ - type: dot_ndcg@10
916
+ value: 0.5024501622170336
917
+ name: Dot Ndcg@10
918
+ - type: dot_mrr@10
919
+ value: 0.45037301587301587
920
+ name: Dot Mrr@10
921
+ - type: dot_map@100
922
+ value: 0.444050525697599
923
+ name: Dot Map@100
924
+ - type: query_active_dims
925
+ value: 51.31999969482422
926
+ name: Query Active Dims
927
+ - type: query_sparsity_ratio
928
+ value: 0.9983185898796008
929
+ name: Query Sparsity Ratio
930
+ - type: corpus_active_dims
931
+ value: 59.146453857421875
932
+ name: Corpus Active Dims
933
+ - type: corpus_sparsity_ratio
934
+ value: 0.998062169783847
935
+ name: Corpus Sparsity Ratio
936
+ - task:
937
+ type: sparse-information-retrieval
938
+ name: Sparse Information Retrieval
939
+ dataset:
940
+ name: NanoSCIDOCS
941
+ type: NanoSCIDOCS
942
+ metrics:
943
+ - type: dot_accuracy@1
944
+ value: 0.28
945
+ name: Dot Accuracy@1
946
+ - type: dot_accuracy@3
947
+ value: 0.58
948
+ name: Dot Accuracy@3
949
+ - type: dot_accuracy@5
950
+ value: 0.62
951
+ name: Dot Accuracy@5
952
+ - type: dot_accuracy@10
953
+ value: 0.8
954
+ name: Dot Accuracy@10
955
+ - type: dot_precision@1
956
+ value: 0.28
957
+ name: Dot Precision@1
958
+ - type: dot_precision@3
959
+ value: 0.24
960
+ name: Dot Precision@3
961
+ - type: dot_precision@5
962
+ value: 0.17199999999999996
963
+ name: Dot Precision@5
964
+ - type: dot_precision@10
965
+ value: 0.13599999999999998
966
+ name: Dot Precision@10
967
+ - type: dot_recall@1
968
+ value: 0.05866666666666667
969
+ name: Dot Recall@1
970
+ - type: dot_recall@3
971
+ value: 0.14766666666666667
972
+ name: Dot Recall@3
973
+ - type: dot_recall@5
974
+ value: 0.17566666666666664
975
+ name: Dot Recall@5
976
+ - type: dot_recall@10
977
+ value: 0.2796666666666667
978
+ name: Dot Recall@10
979
+ - type: dot_ndcg@10
980
+ value: 0.25565589285716384
981
+ name: Dot Ndcg@10
982
+ - type: dot_mrr@10
983
+ value: 0.4341031746031745
984
+ name: Dot Mrr@10
985
+ - type: dot_map@100
986
+ value: 0.16804725663907635
987
+ name: Dot Map@100
988
+ - type: query_active_dims
989
+ value: 195.27999877929688
990
+ name: Query Active Dims
991
+ - type: query_sparsity_ratio
992
+ value: 0.9936019920457605
993
+ name: Query Sparsity Ratio
994
+ - type: corpus_active_dims
995
+ value: 1035.02685546875
996
+ name: Corpus Active Dims
997
+ - type: corpus_sparsity_ratio
998
+ value: 0.9660891535460078
999
+ name: Corpus Sparsity Ratio
1000
+ - task:
1001
+ type: sparse-information-retrieval
1002
+ name: Sparse Information Retrieval
1003
+ dataset:
1004
+ name: NanoArguAna
1005
+ type: NanoArguAna
1006
+ metrics:
1007
+ - type: dot_accuracy@1
1008
+ value: 0.02
1009
+ name: Dot Accuracy@1
1010
+ - type: dot_accuracy@3
1011
+ value: 0.12
1012
+ name: Dot Accuracy@3
1013
+ - type: dot_accuracy@5
1014
+ value: 0.14
1015
+ name: Dot Accuracy@5
1016
+ - type: dot_accuracy@10
1017
+ value: 0.16
1018
+ name: Dot Accuracy@10
1019
+ - type: dot_precision@1
1020
+ value: 0.02
1021
+ name: Dot Precision@1
1022
+ - type: dot_precision@3
1023
+ value: 0.039999999999999994
1024
+ name: Dot Precision@3
1025
+ - type: dot_precision@5
1026
+ value: 0.028000000000000004
1027
+ name: Dot Precision@5
1028
+ - type: dot_precision@10
1029
+ value: 0.016
1030
+ name: Dot Precision@10
1031
+ - type: dot_recall@1
1032
+ value: 0.02
1033
+ name: Dot Recall@1
1034
+ - type: dot_recall@3
1035
+ value: 0.12
1036
+ name: Dot Recall@3
1037
+ - type: dot_recall@5
1038
+ value: 0.14
1039
+ name: Dot Recall@5
1040
+ - type: dot_recall@10
1041
+ value: 0.16
1042
+ name: Dot Recall@10
1043
+ - type: dot_ndcg@10
1044
+ value: 0.09097486504648661
1045
+ name: Dot Ndcg@10
1046
+ - type: dot_mrr@10
1047
+ value: 0.06833333333333333
1048
+ name: Dot Mrr@10
1049
+ - type: dot_map@100
1050
+ value: 0.07512669033130494
1051
+ name: Dot Map@100
1052
+ - type: query_active_dims
1053
+ value: 1119.800048828125
1054
+ name: Query Active Dims
1055
+ - type: query_sparsity_ratio
1056
+ value: 0.9633117079867596
1057
+ name: Query Sparsity Ratio
1058
+ - type: corpus_active_dims
1059
+ value: 936.6198120117188
1060
+ name: Corpus Active Dims
1061
+ - type: corpus_sparsity_ratio
1062
+ value: 0.9693132883817667
1063
+ name: Corpus Sparsity Ratio
1064
+ - task:
1065
+ type: sparse-information-retrieval
1066
+ name: Sparse Information Retrieval
1067
+ dataset:
1068
+ name: NanoSciFact
1069
+ type: NanoSciFact
1070
+ metrics:
1071
+ - type: dot_accuracy@1
1072
+ value: 0.36
1073
+ name: Dot Accuracy@1
1074
+ - type: dot_accuracy@3
1075
+ value: 0.54
1076
+ name: Dot Accuracy@3
1077
+ - type: dot_accuracy@5
1078
+ value: 0.58
1079
+ name: Dot Accuracy@5
1080
+ - type: dot_accuracy@10
1081
+ value: 0.64
1082
+ name: Dot Accuracy@10
1083
+ - type: dot_precision@1
1084
+ value: 0.36
1085
+ name: Dot Precision@1
1086
+ - type: dot_precision@3
1087
+ value: 0.19333333333333333
1088
+ name: Dot Precision@3
1089
+ - type: dot_precision@5
1090
+ value: 0.124
1091
+ name: Dot Precision@5
1092
+ - type: dot_precision@10
1093
+ value: 0.074
1094
+ name: Dot Precision@10
1095
+ - type: dot_recall@1
1096
+ value: 0.335
1097
+ name: Dot Recall@1
1098
+ - type: dot_recall@3
1099
+ value: 0.515
1100
+ name: Dot Recall@3
1101
+ - type: dot_recall@5
1102
+ value: 0.545
1103
+ name: Dot Recall@5
1104
+ - type: dot_recall@10
1105
+ value: 0.63
1106
+ name: Dot Recall@10
1107
+ - type: dot_ndcg@10
1108
+ value: 0.4915918543191975
1109
+ name: Dot Ndcg@10
1110
+ - type: dot_mrr@10
1111
+ value: 0.4533333333333332
1112
+ name: Dot Mrr@10
1113
+ - type: dot_map@100
1114
+ value: 0.4491987141282297
1115
+ name: Dot Map@100
1116
+ - type: query_active_dims
1117
+ value: 299.3399963378906
1118
+ name: Query Active Dims
1119
+ - type: query_sparsity_ratio
1120
+ value: 0.9901926480460688
1121
+ name: Query Sparsity Ratio
1122
+ - type: corpus_active_dims
1123
+ value: 1136.7972412109375
1124
+ name: Corpus Active Dims
1125
+ - type: corpus_sparsity_ratio
1126
+ value: 0.9627548246769236
1127
+ name: Corpus Sparsity Ratio
1128
+ - task:
1129
+ type: sparse-information-retrieval
1130
+ name: Sparse Information Retrieval
1131
+ dataset:
1132
+ name: NanoTouche2020
1133
+ type: NanoTouche2020
1134
+ metrics:
1135
+ - type: dot_accuracy@1
1136
+ value: 0.5714285714285714
1137
+ name: Dot Accuracy@1
1138
+ - type: dot_accuracy@3
1139
+ value: 0.8163265306122449
1140
+ name: Dot Accuracy@3
1141
+ - type: dot_accuracy@5
1142
+ value: 0.9591836734693877
1143
+ name: Dot Accuracy@5
1144
+ - type: dot_accuracy@10
1145
+ value: 1.0
1146
+ name: Dot Accuracy@10
1147
+ - type: dot_precision@1
1148
+ value: 0.5714285714285714
1149
+ name: Dot Precision@1
1150
+ - type: dot_precision@3
1151
+ value: 0.5578231292517006
1152
+ name: Dot Precision@3
1153
+ - type: dot_precision@5
1154
+ value: 0.5387755102040817
1155
+ name: Dot Precision@5
1156
+ - type: dot_precision@10
1157
+ value: 0.4265306122448979
1158
+ name: Dot Precision@10
1159
+ - type: dot_recall@1
1160
+ value: 0.03907945255462338
1161
+ name: Dot Recall@1
1162
+ - type: dot_recall@3
1163
+ value: 0.1141786135299426
1164
+ name: Dot Recall@3
1165
+ - type: dot_recall@5
1166
+ value: 0.17607960990710933
1167
+ name: Dot Recall@5
1168
+ - type: dot_recall@10
1169
+ value: 0.26785623174003165
1170
+ name: Dot Recall@10
1171
+ - type: dot_ndcg@10
1172
+ value: 0.4784358025208683
1173
+ name: Dot Ndcg@10
1174
+ - type: dot_mrr@10
1175
+ value: 0.7194120505344994
1176
+ name: Dot Mrr@10
1177
+ - type: dot_map@100
1178
+ value: 0.3382724018630733
1179
+ name: Dot Map@100
1180
+ - type: query_active_dims
1181
+ value: 39.1020393371582
1182
+ name: Query Active Dims
1183
+ - type: query_sparsity_ratio
1184
+ value: 0.9987188900027142
1185
+ name: Query Sparsity Ratio
1186
+ - type: corpus_active_dims
1187
+ value: 630.3636474609375
1188
+ name: Corpus Active Dims
1189
+ - type: corpus_sparsity_ratio
1190
+ value: 0.9793472365028197
1191
+ name: Corpus Sparsity Ratio
1192
+ ---
1193
+
1194
+ # splade-distilbert-base-uncased trained on GooAQ
1195
+
1196
+ This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
1197
+ ## Model Details
1198
+
1199
+ ### Model Description
1200
+ - **Model Type:** SPLADE Sparse Encoder
1201
+ - **Base model:** [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) <!-- at revision 12040accade4e8a0f71eabdb258fecc2e7e948be -->
1202
+ - **Maximum Sequence Length:** 256 tokens
1203
+ - **Output Dimensionality:** 30522 dimensions
1204
+ - **Similarity Function:** Dot Product
1205
+ - **Training Dataset:**
1206
+ - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
1207
+ - **Language:** en
1208
+ - **License:** apache-2.0
1209
+
1210
+ ### Model Sources
1211
+
1212
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
1213
+ - **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
1214
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
1215
+ - **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
1216
+
1217
+ ### Full Model Architecture
1218
+
1219
+ ```
1220
+ SparseEncoder(
1221
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'DistilBertForMaskedLM'})
1222
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
1223
+ )
1224
+ ```
1225
+
1226
+ ## Usage
1227
+
1228
+ ### Direct Usage (Sentence Transformers)
1229
+
1230
+ First install the Sentence Transformers library:
1231
+
1232
+ ```bash
1233
+ pip install -U sentence-transformers
1234
+ ```
1235
+
1236
+ Then you can load this model and run inference.
1237
+ ```python
1238
+ from sentence_transformers import SparseEncoder
1239
+
1240
+ # Download from the 🤗 Hub
1241
+ model = SparseEncoder("tomaarsen/splade-distilbert-base-uncased-gooaq-peft")
1242
+ # Run inference
1243
+ queries = [
1244
+ "how many days for doxycycline to work on sinus infection?",
1245
+ ]
1246
+ documents = [
1247
+ 'Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks for chronic sinusitis.',
1248
+ 'Most engagements typically have a cocktail dress code, calling for dresses at, or slightly above, knee-length and high heels. If your party states a different dress code, however, such as semi-formal or dressy-casual, you may need to dress up or down accordingly.',
1249
+ 'The average service life of a gas furnace is about 15 years, but the actual life span of an individual unit can vary greatly. There are a number of contributing factors that determine the age a furnace reaches: The quality of the equipment.',
1250
+ ]
1251
+ query_embeddings = model.encode_query(queries)
1252
+ document_embeddings = model.encode_document(documents)
1253
+ print(query_embeddings.shape, document_embeddings.shape)
1254
+ # [1, 30522] [3, 30522]
1255
+
1256
+ # Get the similarity scores for the embeddings
1257
+ similarities = model.similarity(query_embeddings, document_embeddings)
1258
+ print(similarities)
1259
+ # tensor([[93.4242, 28.8323, 33.3142]])
1260
+ ```
1261
+
1262
+ <!--
1263
+ ### Direct Usage (Transformers)
1264
+
1265
+ <details><summary>Click to see the direct usage in Transformers</summary>
1266
+
1267
+ </details>
1268
+ -->
1269
+
1270
+ <!--
1271
+ ### Downstream Usage (Sentence Transformers)
1272
+
1273
+ You can finetune this model on your own dataset.
1274
+
1275
+ <details><summary>Click to expand</summary>
1276
+
1277
+ </details>
1278
+ -->
1279
+
1280
+ <!--
1281
+ ### Out-of-Scope Use
1282
+
1283
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
1284
+ -->
1285
+
1286
+ ## Evaluation
1287
+
1288
+ ### Metrics
1289
+
1290
+ #### Sparse Information Retrieval
1291
+
1292
+ * Datasets: `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
1293
+ * Evaluated with [<code>SparseInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator)
1294
+
1295
+ | Metric | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
1296
+ |:----------------------|:------------|:-------------|:-----------|:-----------------|:------------|:-----------|:-------------|:-------------|:-------------------|:------------|:------------|:------------|:---------------|
1297
+ | dot_accuracy@1 | 0.22 | 0.28 | 0.22 | 0.22 | 0.52 | 0.58 | 0.14 | 0.48 | 0.34 | 0.28 | 0.02 | 0.36 | 0.5714 |
1298
+ | dot_accuracy@3 | 0.4 | 0.42 | 0.36 | 0.28 | 0.74 | 0.66 | 0.26 | 0.64 | 0.48 | 0.58 | 0.12 | 0.54 | 0.8163 |
1299
+ | dot_accuracy@5 | 0.5 | 0.44 | 0.5 | 0.36 | 0.84 | 0.72 | 0.36 | 0.78 | 0.58 | 0.62 | 0.14 | 0.58 | 0.9592 |
1300
+ | dot_accuracy@10 | 0.7 | 0.48 | 0.56 | 0.46 | 0.88 | 0.88 | 0.46 | 0.84 | 0.74 | 0.8 | 0.16 | 0.64 | 1.0 |
1301
+ | dot_precision@1 | 0.22 | 0.28 | 0.22 | 0.22 | 0.52 | 0.58 | 0.14 | 0.48 | 0.34 | 0.28 | 0.02 | 0.36 | 0.5714 |
1302
+ | dot_precision@3 | 0.1333 | 0.2533 | 0.12 | 0.1133 | 0.46 | 0.22 | 0.1067 | 0.2667 | 0.16 | 0.24 | 0.04 | 0.1933 | 0.5578 |
1303
+ | dot_precision@5 | 0.1 | 0.208 | 0.1 | 0.084 | 0.452 | 0.148 | 0.104 | 0.204 | 0.12 | 0.172 | 0.028 | 0.124 | 0.5388 |
1304
+ | dot_precision@10 | 0.07 | 0.172 | 0.056 | 0.058 | 0.384 | 0.09 | 0.068 | 0.114 | 0.078 | 0.136 | 0.016 | 0.074 | 0.4265 |
1305
+ | dot_recall@1 | 0.22 | 0.0103 | 0.2 | 0.0917 | 0.0497 | 0.56 | 0.0793 | 0.24 | 0.3267 | 0.0587 | 0.02 | 0.335 | 0.0391 |
1306
+ | dot_recall@3 | 0.4 | 0.0243 | 0.33 | 0.1533 | 0.1035 | 0.63 | 0.157 | 0.4 | 0.4467 | 0.1477 | 0.12 | 0.515 | 0.1142 |
1307
+ | dot_recall@5 | 0.5 | 0.0332 | 0.46 | 0.1767 | 0.1643 | 0.7 | 0.2572 | 0.51 | 0.5407 | 0.1757 | 0.14 | 0.545 | 0.1761 |
1308
+ | dot_recall@10 | 0.7 | 0.0585 | 0.52 | 0.2247 | 0.2407 | 0.8467 | 0.3007 | 0.57 | 0.7107 | 0.2797 | 0.16 | 0.63 | 0.2679 |
1309
+ | **dot_ndcg@10** | **0.4332** | **0.1972** | **0.3557** | **0.1943** | **0.4566** | **0.6815** | **0.2172** | **0.4894** | **0.5025** | **0.2557** | **0.091** | **0.4916** | **0.4784** |
1310
+ | dot_mrr@10 | 0.3512 | 0.3504 | 0.322 | 0.2767 | 0.6437 | 0.6464 | 0.2306 | 0.5976 | 0.4504 | 0.4341 | 0.0683 | 0.4533 | 0.7194 |
1311
+ | dot_map@100 | 0.3625 | 0.0641 | 0.3138 | 0.1549 | 0.3021 | 0.6297 | 0.1718 | 0.4127 | 0.4441 | 0.168 | 0.0751 | 0.4492 | 0.3383 |
1312
+ | query_active_dims | 114.84 | 185.0 | 98.22 | 259.86 | 191.26 | 249.54 | 87.4 | 151.22 | 51.32 | 195.28 | 1119.8 | 299.34 | 39.102 |
1313
+ | query_sparsity_ratio | 0.9962 | 0.9939 | 0.9968 | 0.9915 | 0.9937 | 0.9918 | 0.9971 | 0.995 | 0.9983 | 0.9936 | 0.9633 | 0.9902 | 0.9987 |
1314
+ | corpus_active_dims | 504.951 | 1286.7938 | 841.8668 | 1094.6027 | 809.2099 | 1358.9602 | 517.6329 | 904.4684 | 59.1465 | 1035.0269 | 936.6198 | 1136.7972 | 630.3636 |
1315
+ | corpus_sparsity_ratio | 0.9835 | 0.9578 | 0.9724 | 0.9641 | 0.9735 | 0.9555 | 0.983 | 0.9704 | 0.9981 | 0.9661 | 0.9693 | 0.9628 | 0.9793 |
1316
+
1317
+ #### Sparse Nano BEIR
1318
+
1319
+ * Dataset: `NanoBEIR_mean`
1320
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1321
+ ```json
1322
+ {
1323
+ "dataset_names": [
1324
+ "msmarco",
1325
+ "nfcorpus",
1326
+ "nq"
1327
+ ]
1328
+ }
1329
+ ```
1330
+
1331
+ | Metric | Value |
1332
+ |:----------------------|:-----------|
1333
+ | dot_accuracy@1 | 0.24 |
1334
+ | dot_accuracy@3 | 0.3933 |
1335
+ | dot_accuracy@5 | 0.48 |
1336
+ | dot_accuracy@10 | 0.58 |
1337
+ | dot_precision@1 | 0.24 |
1338
+ | dot_precision@3 | 0.1689 |
1339
+ | dot_precision@5 | 0.136 |
1340
+ | dot_precision@10 | 0.0993 |
1341
+ | dot_recall@1 | 0.1434 |
1342
+ | dot_recall@3 | 0.2514 |
1343
+ | dot_recall@5 | 0.3311 |
1344
+ | dot_recall@10 | 0.4262 |
1345
+ | **dot_ndcg@10** | **0.3287** |
1346
+ | dot_mrr@10 | 0.3412 |
1347
+ | dot_map@100 | 0.2468 |
1348
+ | query_active_dims | 132.6867 |
1349
+ | query_sparsity_ratio | 0.9957 |
1350
+ | corpus_active_dims | 812.3068 |
1351
+ | corpus_sparsity_ratio | 0.9734 |
1352
+
1353
+ #### Sparse Nano BEIR
1354
+
1355
+ * Dataset: `NanoBEIR_mean`
1356
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1357
+ ```json
1358
+ {
1359
+ "dataset_names": [
1360
+ "climatefever",
1361
+ "dbpedia",
1362
+ "fever",
1363
+ "fiqa2018",
1364
+ "hotpotqa",
1365
+ "msmarco",
1366
+ "nfcorpus",
1367
+ "nq",
1368
+ "quoraretrieval",
1369
+ "scidocs",
1370
+ "arguana",
1371
+ "scifact",
1372
+ "touche2020"
1373
+ ]
1374
+ }
1375
+ ```
1376
+
1377
+ | Metric | Value |
1378
+ |:----------------------|:-----------|
1379
+ | dot_accuracy@1 | 0.3255 |
1380
+ | dot_accuracy@3 | 0.4843 |
1381
+ | dot_accuracy@5 | 0.5676 |
1382
+ | dot_accuracy@10 | 0.6615 |
1383
+ | dot_precision@1 | 0.3255 |
1384
+ | dot_precision@3 | 0.2203 |
1385
+ | dot_precision@5 | 0.1833 |
1386
+ | dot_precision@10 | 0.134 |
1387
+ | dot_recall@1 | 0.1716 |
1388
+ | dot_recall@3 | 0.2724 |
1389
+ | dot_recall@5 | 0.3368 |
1390
+ | dot_recall@10 | 0.4238 |
1391
+ | **dot_ndcg@10** | **0.3726** |
1392
+ | dot_mrr@10 | 0.4265 |
1393
+ | dot_map@100 | 0.2989 |
1394
+ | query_active_dims | 234.3143 |
1395
+ | query_sparsity_ratio | 0.9923 |
1396
+ | corpus_active_dims | 808.1458 |
1397
+ | corpus_sparsity_ratio | 0.9735 |
1398
+
1399
+ <!--
1400
+ ## Bias, Risks and Limitations
1401
+
1402
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
1403
+ -->
1404
+
1405
+ <!--
1406
+ ### Recommendations
1407
+
1408
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
1409
+ -->
1410
+
1411
+ ## Training Details
1412
+
1413
+ ### Training Dataset
1414
+
1415
+ #### gooaq
1416
+
1417
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1418
+ * Size: 99,000 training samples
1419
+ * Columns: <code>question</code> and <code>answer</code>
1420
+ * Approximate statistics based on the first 1000 samples:
1421
+ | | question | answer |
1422
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1423
+ | type | string | string |
1424
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.79 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.02 tokens</li><li>max: 153 tokens</li></ul> |
1425
+ * Samples:
1426
+ | question | answer |
1427
+ |:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1428
+ | <code>what are the 5 characteristics of a star?</code> | <code>Key Concept: Characteristics used to classify stars include color, temperature, size, composition, and brightness.</code> |
1429
+ | <code>are copic markers alcohol ink?</code> | <code>Copic Ink is alcohol-based and flammable. Keep away from direct sunlight and extreme temperatures.</code> |
1430
+ | <code>what is the difference between appellate term and appellate division?</code> | <code>Appellate terms An appellate term is an intermediate appellate court that hears appeals from the inferior courts within their designated counties or judicial districts, and are intended to ease the workload on the Appellate Division and provide a less expensive forum closer to the people.</code> |
1431
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1432
+ ```json
1433
+ {
1434
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1435
+ "document_regularizer_weight": 3e-05,
1436
+ "query_regularizer_weight": 5e-05
1437
+ }
1438
+ ```
1439
+
1440
+ ### Evaluation Dataset
1441
+
1442
+ #### gooaq
1443
+
1444
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1445
+ * Size: 1,000 evaluation samples
1446
+ * Columns: <code>question</code> and <code>answer</code>
1447
+ * Approximate statistics based on the first 1000 samples:
1448
+ | | question | answer |
1449
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1450
+ | type | string | string |
1451
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.93 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.84 tokens</li><li>max: 127 tokens</li></ul> |
1452
+ * Samples:
1453
+ | question | answer |
1454
+ |:-----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1455
+ | <code>should you take ibuprofen with high blood pressure?</code> | <code>In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor.</code> |
1456
+ | <code>how old do you have to be to work in sc?</code> | <code>The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor.</code> |
1457
+ | <code>how to write a topic proposal for a research paper?</code> | <code>['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.']</code> |
1458
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1459
+ ```json
1460
+ {
1461
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1462
+ "document_regularizer_weight": 3e-05,
1463
+ "query_regularizer_weight": 5e-05
1464
+ }
1465
+ ```
1466
+
1467
+ ### Training Hyperparameters
1468
+ #### Non-Default Hyperparameters
1469
+
1470
+ - `eval_strategy`: steps
1471
+ - `per_device_train_batch_size`: 32
1472
+ - `per_device_eval_batch_size`: 32
1473
+ - `learning_rate`: 2e-05
1474
+ - `num_train_epochs`: 1
1475
+ - `bf16`: True
1476
+ - `load_best_model_at_end`: True
1477
+ - `batch_sampler`: no_duplicates
1478
+
1479
+ #### All Hyperparameters
1480
+ <details><summary>Click to expand</summary>
1481
+
1482
+ - `overwrite_output_dir`: False
1483
+ - `do_predict`: False
1484
+ - `eval_strategy`: steps
1485
+ - `prediction_loss_only`: True
1486
+ - `per_device_train_batch_size`: 32
1487
+ - `per_device_eval_batch_size`: 32
1488
+ - `per_gpu_train_batch_size`: None
1489
+ - `per_gpu_eval_batch_size`: None
1490
+ - `gradient_accumulation_steps`: 1
1491
+ - `eval_accumulation_steps`: None
1492
+ - `torch_empty_cache_steps`: None
1493
+ - `learning_rate`: 2e-05
1494
+ - `weight_decay`: 0.0
1495
+ - `adam_beta1`: 0.9
1496
+ - `adam_beta2`: 0.999
1497
+ - `adam_epsilon`: 1e-08
1498
+ - `max_grad_norm`: 1.0
1499
+ - `num_train_epochs`: 1
1500
+ - `max_steps`: -1
1501
+ - `lr_scheduler_type`: linear
1502
+ - `lr_scheduler_kwargs`: {}
1503
+ - `warmup_ratio`: 0.0
1504
+ - `warmup_steps`: 0
1505
+ - `log_level`: passive
1506
+ - `log_level_replica`: warning
1507
+ - `log_on_each_node`: True
1508
+ - `logging_nan_inf_filter`: True
1509
+ - `save_safetensors`: True
1510
+ - `save_on_each_node`: False
1511
+ - `save_only_model`: False
1512
+ - `restore_callback_states_from_checkpoint`: False
1513
+ - `no_cuda`: False
1514
+ - `use_cpu`: False
1515
+ - `use_mps_device`: False
1516
+ - `seed`: 42
1517
+ - `data_seed`: None
1518
+ - `jit_mode_eval`: False
1519
+ - `use_ipex`: False
1520
+ - `bf16`: True
1521
+ - `fp16`: False
1522
+ - `fp16_opt_level`: O1
1523
+ - `half_precision_backend`: auto
1524
+ - `bf16_full_eval`: False
1525
+ - `fp16_full_eval`: False
1526
+ - `tf32`: None
1527
+ - `local_rank`: 0
1528
+ - `ddp_backend`: None
1529
+ - `tpu_num_cores`: None
1530
+ - `tpu_metrics_debug`: False
1531
+ - `debug`: []
1532
+ - `dataloader_drop_last`: False
1533
+ - `dataloader_num_workers`: 0
1534
+ - `dataloader_prefetch_factor`: None
1535
+ - `past_index`: -1
1536
+ - `disable_tqdm`: False
1537
+ - `remove_unused_columns`: True
1538
+ - `label_names`: None
1539
+ - `load_best_model_at_end`: True
1540
+ - `ignore_data_skip`: False
1541
+ - `fsdp`: []
1542
+ - `fsdp_min_num_params`: 0
1543
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
1544
+ - `fsdp_transformer_layer_cls_to_wrap`: None
1545
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
1546
+ - `deepspeed`: None
1547
+ - `label_smoothing_factor`: 0.0
1548
+ - `optim`: adamw_torch
1549
+ - `optim_args`: None
1550
+ - `adafactor`: False
1551
+ - `group_by_length`: False
1552
+ - `length_column_name`: length
1553
+ - `ddp_find_unused_parameters`: None
1554
+ - `ddp_bucket_cap_mb`: None
1555
+ - `ddp_broadcast_buffers`: False
1556
+ - `dataloader_pin_memory`: True
1557
+ - `dataloader_persistent_workers`: False
1558
+ - `skip_memory_metrics`: True
1559
+ - `use_legacy_prediction_loop`: False
1560
+ - `push_to_hub`: False
1561
+ - `resume_from_checkpoint`: None
1562
+ - `hub_model_id`: None
1563
+ - `hub_strategy`: every_save
1564
+ - `hub_private_repo`: None
1565
+ - `hub_always_push`: False
1566
+ - `gradient_checkpointing`: False
1567
+ - `gradient_checkpointing_kwargs`: None
1568
+ - `include_inputs_for_metrics`: False
1569
+ - `include_for_metrics`: []
1570
+ - `eval_do_concat_batches`: True
1571
+ - `fp16_backend`: auto
1572
+ - `push_to_hub_model_id`: None
1573
+ - `push_to_hub_organization`: None
1574
+ - `mp_parameters`:
1575
+ - `auto_find_batch_size`: False
1576
+ - `full_determinism`: False
1577
+ - `torchdynamo`: None
1578
+ - `ray_scope`: last
1579
+ - `ddp_timeout`: 1800
1580
+ - `torch_compile`: False
1581
+ - `torch_compile_backend`: None
1582
+ - `torch_compile_mode`: None
1583
+ - `include_tokens_per_second`: False
1584
+ - `include_num_input_tokens_seen`: False
1585
+ - `neftune_noise_alpha`: None
1586
+ - `optim_target_modules`: None
1587
+ - `batch_eval_metrics`: False
1588
+ - `eval_on_start`: False
1589
+ - `use_liger_kernel`: False
1590
+ - `eval_use_gather_object`: False
1591
+ - `average_tokens_across_devices`: False
1592
+ - `prompts`: None
1593
+ - `batch_sampler`: no_duplicates
1594
+ - `multi_dataset_batch_sampler`: proportional
1595
+ - `router_mapping`: {}
1596
+ - `learning_rate_mapping`: {}
1597
+
1598
+ </details>
1599
+
1600
+ ### Training Logs
1601
+ | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 |
1602
+ |:----------:|:--------:|:-------------:|:---------------:|:-----------------------:|:------------------------:|:------------------:|:-------------------------:|:----------------------------:|:-----------------------:|:---------------------:|:------------------------:|:------------------------:|:------------------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:--------------------------:|
1603
+ | 0.0323 | 100 | 234.4946 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1604
+ | 0.0646 | 200 | 90.2538 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1605
+ | 0.0970 | 300 | 35.2404 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1606
+ | 0.1293 | 400 | 15.0794 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1607
+ | 0.1616 | 500 | 5.7405 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1608
+ | 0.1939 | 600 | 2.6706 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1609
+ | 0.1972 | 610 | - | 1.5711 | 0.1942 | 0.1431 | 0.1568 | 0.1647 | - | - | - | - | - | - | - | - | - | - |
1610
+ | 0.2262 | 700 | 1.4867 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1611
+ | 0.2586 | 800 | 0.9108 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1612
+ | 0.2909 | 900 | 0.7938 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1613
+ | 0.3232 | 1000 | 0.6679 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1614
+ | 0.3555 | 1100 | 0.5505 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1615
+ | 0.3878 | 1200 | 0.4851 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1616
+ | 0.3943 | 1220 | - | 0.3510 | 0.3406 | 0.1831 | 0.2740 | 0.2659 | - | - | - | - | - | - | - | - | - | - |
1617
+ | 0.4202 | 1300 | 0.4882 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1618
+ | 0.4525 | 1400 | 0.4156 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1619
+ | 0.4848 | 1500 | 0.452 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1620
+ | 0.5171 | 1600 | 0.3446 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1621
+ | 0.5495 | 1700 | 0.307 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1622
+ | 0.5818 | 1800 | 0.3416 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1623
+ | 0.5915 | 1830 | - | 0.2682 | 0.3942 | 0.1917 | 0.3140 | 0.3000 | - | - | - | - | - | - | - | - | - | - |
1624
+ | 0.6141 | 1900 | 0.2875 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1625
+ | 0.6464 | 2000 | 0.2989 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1626
+ | 0.6787 | 2100 | 0.3032 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1627
+ | 0.7111 | 2200 | 0.3843 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1628
+ | 0.7434 | 2300 | 0.2845 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1629
+ | 0.7757 | 2400 | 0.2838 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1630
+ | 0.7886 | 2440 | - | 0.2365 | 0.4144 | 0.1952 | 0.3378 | 0.3158 | - | - | - | - | - | - | - | - | - | - |
1631
+ | 0.8080 | 2500 | 0.2422 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1632
+ | 0.8403 | 2600 | 0.2546 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1633
+ | 0.8727 | 2700 | 0.2683 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1634
+ | 0.9050 | 2800 | 0.2923 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1635
+ | 0.9373 | 2900 | 0.301 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1636
+ | 0.9696 | 3000 | 0.2796 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1637
+ | **0.9858** | **3050** | **-** | **0.2284** | **0.4332** | **0.1972** | **0.3557** | **0.3287** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** |
1638
+ | -1 | -1 | - | - | 0.4332 | 0.1972 | 0.3557 | 0.3726 | 0.1943 | 0.4566 | 0.6815 | 0.2172 | 0.4894 | 0.5025 | 0.2557 | 0.0910 | 0.4916 | 0.4784 |
1639
+
1640
+ * The bold row denotes the saved checkpoint.
1641
+
1642
+ ### Environmental Impact
1643
+ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
1644
+ - **Energy Consumed**: 0.034 kWh
1645
+ - **Carbon Emitted**: 0.013 kg of CO2
1646
+ - **Hours Used**: 0.145 hours
1647
+
1648
+ ### Training Hardware
1649
+ - **On Cloud**: No
1650
+ - **GPU Model**: 1 x NVIDIA GeForce RTX 3090
1651
+ - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
1652
+ - **RAM Size**: 31.78 GB
1653
+
1654
+ ### Framework Versions
1655
+ - Python: 3.11.6
1656
+ - Sentence Transformers: 4.2.0.dev0
1657
+ - Transformers: 4.52.4
1658
+ - PyTorch: 2.7.1+cu126
1659
+ - Accelerate: 1.5.1
1660
+ - Datasets: 2.21.0
1661
+ - Tokenizers: 0.21.1
1662
+
1663
+ ## Citation
1664
+
1665
+ ### BibTeX
1666
+
1667
+ #### Sentence Transformers
1668
+ ```bibtex
1669
+ @inproceedings{reimers-2019-sentence-bert,
1670
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
1671
+ author = "Reimers, Nils and Gurevych, Iryna",
1672
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
1673
+ month = "11",
1674
+ year = "2019",
1675
+ publisher = "Association for Computational Linguistics",
1676
+ url = "https://arxiv.org/abs/1908.10084",
1677
+ }
1678
+ ```
1679
+
1680
+ #### SpladeLoss
1681
+ ```bibtex
1682
+ @misc{formal2022distillationhardnegativesampling,
1683
+ title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
1684
+ author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
1685
+ year={2022},
1686
+ eprint={2205.04733},
1687
+ archivePrefix={arXiv},
1688
+ primaryClass={cs.IR},
1689
+ url={https://arxiv.org/abs/2205.04733},
1690
+ }
1691
+ ```
1692
+
1693
+ #### SparseMultipleNegativesRankingLoss
1694
+ ```bibtex
1695
+ @misc{henderson2017efficient,
1696
+ title={Efficient Natural Language Response Suggestion for Smart Reply},
1697
+ author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
1698
+ year={2017},
1699
+ eprint={1705.00652},
1700
+ archivePrefix={arXiv},
1701
+ primaryClass={cs.CL}
1702
+ }
1703
+ ```
1704
+
1705
+ #### FlopsLoss
1706
+ ```bibtex
1707
+ @article{paria2020minimizing,
1708
+ title={Minimizing flops to learn efficient sparse representations},
1709
+ author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s},
1710
+ journal={arXiv preprint arXiv:2004.05665},
1711
+ year={2020}
1712
+ }
1713
+ ```
1714
+
1715
+ <!--
1716
+ ## Glossary
1717
+
1718
+ *Clearly define terms in order to be accessible across audiences.*
1719
+ -->
1720
+
1721
+ <!--
1722
+ ## Model Card Authors
1723
+
1724
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
1725
+ -->
1726
+
1727
+ <!--
1728
+ ## Model Card Contact
1729
+
1730
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
1731
+ -->
adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "distilbert/distilbert-base-uncased",
5
+ "bias": "none",
6
+ "corda_config": null,
7
+ "eva_config": null,
8
+ "exclude_modules": null,
9
+ "fan_in_fan_out": false,
10
+ "inference_mode": false,
11
+ "init_lora_weights": true,
12
+ "layer_replication": null,
13
+ "layers_pattern": null,
14
+ "layers_to_transform": null,
15
+ "loftq_config": {},
16
+ "lora_alpha": 32,
17
+ "lora_bias": false,
18
+ "lora_dropout": 0.05,
19
+ "megatron_config": null,
20
+ "megatron_core": "megatron.core",
21
+ "modules_to_save": null,
22
+ "peft_type": "LORA",
23
+ "r": 16,
24
+ "rank_pattern": {},
25
+ "revision": null,
26
+ "target_modules": [
27
+ "q_lin",
28
+ "v_lin"
29
+ ],
30
+ "task_type": "FEATURE_EXTRACTION",
31
+ "trainable_token_indices": null,
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3293092d3bebb0f87dcc1cf412743503ca4fefa70cb58b18c6da5626e883c2c
3
+ size 1183144
config_sentence_transformers.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "SparseEncoder",
3
+ "__version__": {
4
+ "sentence_transformers": "4.2.0.dev0",
5
+ "transformers": "4.52.4",
6
+ "pytorch": "2.7.1+cu126"
7
+ },
8
+ "prompts": {
9
+ "query": "",
10
+ "document": ""
11
+ },
12
+ "default_prompt_name": null,
13
+ "similarity_fn_name": "dot"
14
+ }
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.sparse_encoder.models.MLMTransformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_SpladePooling",
12
+ "type": "sentence_transformers.sparse_encoder.models.SpladePooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 256,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": true,
47
+ "extra_special_tokens": {},
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "pad_token": "[PAD]",
51
+ "sep_token": "[SEP]",
52
+ "strip_accents": null,
53
+ "tokenize_chinese_chars": true,
54
+ "tokenizer_class": "DistilBertTokenizer",
55
+ "unk_token": "[UNK]"
56
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff