metadata
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sparse-encoder
- sparse
- splade
- generated_from_trainer
- dataset_size:99000
- loss:SpladeLoss
- loss:SparseMultipleNegativesRankingLoss
- loss:FlopsLoss
base_model: google-bert/bert-base-uncased
widget:
- text: >-
The term emergent literacy signals a belief that, in a literate society,
young children even one and two year olds, are in the process of becoming
literate”. ... Gray (1956:21) notes: Functional literacy is used for the
training of adults to 'meet independently the reading and writing demands
placed on them'.
- text: >-
Rey is seemingly confirmed as being The Chosen One per a quote by a
Lucasfilm production designer who worked on The Rise of Skywalker.
- text: are union gun safes fireproof?
- text: >-
Fruit is an essential part of a healthy diet — and may aid weight loss.
Most fruits are low in calories while high in nutrients and fiber, which
can boost your fullness. Keep in mind that it's best to eat fruits whole
rather than juiced. What's more, simply eating fruit is not the key to
weight loss.
- text: >-
Treatment of suspected bacterial infection is with antibiotics, such as
amoxicillin/clavulanate or doxycycline, given for 5 to 7 days for acute
sinusitis and for up to 6 weeks for chronic sinusitis.
datasets:
- sentence-transformers/gooaq
pipeline_tag: feature-extraction
library_name: sentence-transformers
metrics:
- dot_accuracy@1
- dot_accuracy@3
- dot_accuracy@5
- dot_accuracy@10
- dot_precision@1
- dot_precision@3
- dot_precision@5
- dot_precision@10
- dot_recall@1
- dot_recall@3
- dot_recall@5
- dot_recall@10
- dot_ndcg@10
- dot_mrr@10
- dot_map@100
- query_active_dims
- query_sparsity_ratio
- corpus_active_dims
- corpus_sparsity_ratio
co2_eq_emissions:
emissions: 22.555839031832004
energy_consumed: 0.058028615833805856
source: codecarbon
training_type: fine-tuning
on_cloud: false
cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
ram_total_size: 31.777088165283203
hours_used: 0.232
hardware_used: 1 x NVIDIA GeForce RTX 3090
model-index:
- name: bert-base-uncased adapter finetuned on GooAQ pairs
results:
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO
type: NanoMSMARCO
metrics:
- type: dot_accuracy@1
value: 0.16
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.4
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.52
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.6
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.16
name: Dot Precision@1
- type: dot_precision@3
value: 0.13333333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.10400000000000001
name: Dot Precision@5
- type: dot_precision@10
value: 0.06000000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.16
name: Dot Recall@1
- type: dot_recall@3
value: 0.4
name: Dot Recall@3
- type: dot_recall@5
value: 0.52
name: Dot Recall@5
- type: dot_recall@10
value: 0.6
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.38421491435302385
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.3143571428571429
name: Dot Mrr@10
- type: dot_map@100
value: 0.33474113801530114
name: Dot Map@100
- type: query_active_dims
value: 134.27999877929688
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9956005504626402
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 569.0715942382812
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9813553635332455
name: Corpus Sparsity Ratio
- type: dot_accuracy@1
value: 0.18
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.42
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.5
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.66
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.18
name: Dot Precision@1
- type: dot_precision@3
value: 0.13999999999999999
name: Dot Precision@3
- type: dot_precision@5
value: 0.1
name: Dot Precision@5
- type: dot_precision@10
value: 0.066
name: Dot Precision@10
- type: dot_recall@1
value: 0.18
name: Dot Recall@1
- type: dot_recall@3
value: 0.42
name: Dot Recall@3
- type: dot_recall@5
value: 0.5
name: Dot Recall@5
- type: dot_recall@10
value: 0.66
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.40466314485393395
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.32460317460317456
name: Dot Mrr@10
- type: dot_map@100
value: 0.3399207622442916
name: Dot Map@100
- type: query_active_dims
value: 119.91999816894531
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9960710307919224
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 558.7083129882812
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9816948983360106
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus
type: NanoNFCorpus
metrics:
- type: dot_accuracy@1
value: 0.28
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.38
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.48
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.56
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.28
name: Dot Precision@1
- type: dot_precision@3
value: 0.21333333333333332
name: Dot Precision@3
- type: dot_precision@5
value: 0.20799999999999996
name: Dot Precision@5
- type: dot_precision@10
value: 0.18999999999999997
name: Dot Precision@10
- type: dot_recall@1
value: 0.009643842973249836
name: Dot Recall@1
- type: dot_recall@3
value: 0.021145927666063272
name: Dot Recall@3
- type: dot_recall@5
value: 0.03330669347394032
name: Dot Recall@5
- type: dot_recall@10
value: 0.06056852476443145
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.20300898862073832
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.35271428571428565
name: Dot Mrr@10
- type: dot_map@100
value: 0.06784215422201557
name: Dot Map@100
- type: query_active_dims
value: 225.1999969482422
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9926217155838988
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 758.9915161132812
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9751329691333045
name: Corpus Sparsity Ratio
- type: dot_accuracy@1
value: 0.28
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.4
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.5
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.52
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.28
name: Dot Precision@1
- type: dot_precision@3
value: 0.21333333333333332
name: Dot Precision@3
- type: dot_precision@5
value: 0.204
name: Dot Precision@5
- type: dot_precision@10
value: 0.192
name: Dot Precision@10
- type: dot_recall@1
value: 0.009643842973249836
name: Dot Recall@1
- type: dot_recall@3
value: 0.020713027233162838
name: Dot Recall@3
- type: dot_recall@5
value: 0.033373623471745904
name: Dot Recall@5
- type: dot_recall@10
value: 0.057702524164229184
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.20494533130049183
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.3591904761904762
name: Dot Mrr@10
- type: dot_map@100
value: 0.07033841029751142
name: Dot Map@100
- type: query_active_dims
value: 204.0800018310547
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9933136753217006
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 768.1585083007812
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.974832628651439
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ
type: NanoNQ
metrics:
- type: dot_accuracy@1
value: 0.28
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.52
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.62
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.68
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.28
name: Dot Precision@1
- type: dot_precision@3
value: 0.1733333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.12400000000000003
name: Dot Precision@5
- type: dot_precision@10
value: 0.07
name: Dot Precision@10
- type: dot_recall@1
value: 0.27
name: Dot Recall@1
- type: dot_recall@3
value: 0.48
name: Dot Recall@3
- type: dot_recall@5
value: 0.58
name: Dot Recall@5
- type: dot_recall@10
value: 0.64
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.456183710090412
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.41005555555555545
name: Dot Mrr@10
- type: dot_map@100
value: 0.40230708079647753
name: Dot Map@100
- type: query_active_dims
value: 115.80000305175781
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9962060152332167
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 666.47705078125
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9781640439426887
name: Corpus Sparsity Ratio
- type: dot_accuracy@1
value: 0.26
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.46
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.62
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.72
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.26
name: Dot Precision@1
- type: dot_precision@3
value: 0.15333333333333332
name: Dot Precision@3
- type: dot_precision@5
value: 0.12400000000000003
name: Dot Precision@5
- type: dot_precision@10
value: 0.07400000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.25
name: Dot Recall@1
- type: dot_recall@3
value: 0.43
name: Dot Recall@3
- type: dot_recall@5
value: 0.58
name: Dot Recall@5
- type: dot_recall@10
value: 0.68
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.4523767145039781
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.39466666666666655
name: Dot Mrr@10
- type: dot_map@100
value: 0.3832114962839765
name: Dot Map@100
- type: query_active_dims
value: 104.4800033569336
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9965768952441867
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 660.2720947265625
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9783673384861227
name: Corpus Sparsity Ratio
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean
type: NanoBEIR_mean
metrics:
- type: dot_accuracy@1
value: 0.24000000000000002
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.43333333333333335
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.54
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.6133333333333334
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.24000000000000002
name: Dot Precision@1
- type: dot_precision@3
value: 0.17333333333333334
name: Dot Precision@3
- type: dot_precision@5
value: 0.14533333333333331
name: Dot Precision@5
- type: dot_precision@10
value: 0.10666666666666667
name: Dot Precision@10
- type: dot_recall@1
value: 0.14654794765774995
name: Dot Recall@1
- type: dot_recall@3
value: 0.30038197588868776
name: Dot Recall@3
- type: dot_recall@5
value: 0.37776889782464673
name: Dot Recall@5
- type: dot_recall@10
value: 0.4335228415881438
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.347802537688058
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.359042328042328
name: Dot Mrr@10
- type: dot_map@100
value: 0.26829679101126475
name: Dot Map@100
- type: query_active_dims
value: 158.42666625976562
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9948094270932519
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 649.7461397828076
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9787122030082299
name: Corpus Sparsity Ratio
- type: dot_accuracy@1
value: 0.30251177394034545
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.46596546310832027
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.539905808477237
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.6323076923076923
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.30251177394034545
name: Dot Precision@1
- type: dot_precision@3
value: 0.21529042386185243
name: Dot Precision@3
- type: dot_precision@5
value: 0.1783736263736264
name: Dot Precision@5
- type: dot_precision@10
value: 0.13637676609105182
name: Dot Precision@10
- type: dot_recall@1
value: 0.14225413356767205
name: Dot Recall@1
- type: dot_recall@3
value: 0.2584890881168072
name: Dot Recall@3
- type: dot_recall@5
value: 0.3166858200553643
name: Dot Recall@5
- type: dot_recall@10
value: 0.402504217163254
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.352844324981004
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.40464005382372714
name: Dot Mrr@10
- type: dot_map@100
value: 0.28418762466289216
name: Dot Map@100
- type: query_active_dims
value: 274.27427396495096
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9910138826431769
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 613.3641760134007
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9799041944822292
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoClimateFEVER
type: NanoClimateFEVER
metrics:
- type: dot_accuracy@1
value: 0.18
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.32
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.36
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.5
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.18
name: Dot Precision@1
- type: dot_precision@3
value: 0.12
name: Dot Precision@3
- type: dot_precision@5
value: 0.092
name: Dot Precision@5
- type: dot_precision@10
value: 0.07
name: Dot Precision@10
- type: dot_recall@1
value: 0.09666666666666666
name: Dot Recall@1
- type: dot_recall@3
value: 0.15899999999999997
name: Dot Recall@3
- type: dot_recall@5
value: 0.19066666666666665
name: Dot Recall@5
- type: dot_recall@10
value: 0.26566666666666666
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.21060314835282243
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.26110317460317456
name: Dot Mrr@10
- type: dot_map@100
value: 0.16539103175017164
name: Dot Map@100
- type: query_active_dims
value: 464.260009765625
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9847893319649557
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 703.13818359375
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9769629059827747
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoDBPedia
type: NanoDBPedia
metrics:
- type: dot_accuracy@1
value: 0.54
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.7
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.84
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.88
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.54
name: Dot Precision@1
- type: dot_precision@3
value: 0.45333333333333337
name: Dot Precision@3
- type: dot_precision@5
value: 0.45199999999999996
name: Dot Precision@5
- type: dot_precision@10
value: 0.414
name: Dot Precision@10
- type: dot_recall@1
value: 0.04495371619535625
name: Dot Recall@1
- type: dot_recall@3
value: 0.09854649158620489
name: Dot Recall@3
- type: dot_recall@5
value: 0.14811311296402138
name: Dot Recall@5
- type: dot_recall@10
value: 0.2518191059637309
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.47484851291171687
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.657079365079365
name: Dot Mrr@10
- type: dot_map@100
value: 0.333901271745345
name: Dot Map@100
- type: query_active_dims
value: 190.66000366210938
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9937533581134228
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 732.0421752929688
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9760159171976618
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoFEVER
type: NanoFEVER
metrics:
- type: dot_accuracy@1
value: 0.34
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.56
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.58
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.7
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.34
name: Dot Precision@1
- type: dot_precision@3
value: 0.19333333333333336
name: Dot Precision@3
- type: dot_precision@5
value: 0.12000000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.07200000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.32
name: Dot Recall@1
- type: dot_recall@3
value: 0.55
name: Dot Recall@3
- type: dot_recall@5
value: 0.57
name: Dot Recall@5
- type: dot_recall@10
value: 0.6766666666666667
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5066164614256192
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.460547619047619
name: Dot Mrr@10
- type: dot_map@100
value: 0.4600294644045045
name: Dot Map@100
- type: query_active_dims
value: 396.8800048828125
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.986996920094266
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 838.1881713867188
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9725382291007563
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoFiQA2018
type: NanoFiQA2018
metrics:
- type: dot_accuracy@1
value: 0.16
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.28
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.36
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.46
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.16
name: Dot Precision@1
- type: dot_precision@3
value: 0.11999999999999998
name: Dot Precision@3
- type: dot_precision@5
value: 0.11200000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.07600000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.06655555555555555
name: Dot Recall@1
- type: dot_recall@3
value: 0.16722222222222222
name: Dot Recall@3
- type: dot_recall@5
value: 0.2236031746031746
name: Dot Recall@5
- type: dot_recall@10
value: 0.32060317460317456
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.22703772428242097
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.24921428571428572
name: Dot Mrr@10
- type: dot_map@100
value: 0.17973302435608687
name: Dot Map@100
- type: query_active_dims
value: 137.83999633789062
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9954839133628893
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 472.53118896484375
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9845183412304289
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoHotpotQA
type: NanoHotpotQA
metrics:
- type: dot_accuracy@1
value: 0.64
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.78
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.84
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.96
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.64
name: Dot Precision@1
- type: dot_precision@3
value: 0.32
name: Dot Precision@3
- type: dot_precision@5
value: 0.22799999999999998
name: Dot Precision@5
- type: dot_precision@10
value: 0.136
name: Dot Precision@10
- type: dot_recall@1
value: 0.32
name: Dot Recall@1
- type: dot_recall@3
value: 0.48
name: Dot Recall@3
- type: dot_recall@5
value: 0.57
name: Dot Recall@5
- type: dot_recall@10
value: 0.68
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5959463797348153
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.7347142857142858
name: Dot Mrr@10
- type: dot_map@100
value: 0.5051629575345892
name: Dot Map@100
- type: query_active_dims
value: 147.22000122070312
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9951765938922514
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 793.5137329101562
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9740019090193908
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoQuoraRetrieval
type: NanoQuoraRetrieval
metrics:
- type: dot_accuracy@1
value: 0.2
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.36
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.42
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.48
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.2
name: Dot Precision@1
- type: dot_precision@3
value: 0.13333333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.092
name: Dot Precision@5
- type: dot_precision@10
value: 0.052000000000000005
name: Dot Precision@10
- type: dot_recall@1
value: 0.17666666666666667
name: Dot Recall@1
- type: dot_recall@3
value: 0.32733333333333337
name: Dot Recall@3
- type: dot_recall@5
value: 0.38733333333333336
name: Dot Recall@5
- type: dot_recall@10
value: 0.434
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.31674229187216874
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.29157142857142854
name: Dot Mrr@10
- type: dot_map@100
value: 0.29472815560103693
name: Dot Map@100
- type: query_active_dims
value: 77
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9974772295393487
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 72.00594329833984
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9976408510812419
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoSCIDOCS
type: NanoSCIDOCS
metrics:
- type: dot_accuracy@1
value: 0.18
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.38
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.42
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.6
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.18
name: Dot Precision@1
- type: dot_precision@3
value: 0.16666666666666669
name: Dot Precision@3
- type: dot_precision@5
value: 0.11599999999999999
name: Dot Precision@5
- type: dot_precision@10
value: 0.09200000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.036000000000000004
name: Dot Recall@1
- type: dot_recall@3
value: 0.10100000000000002
name: Dot Recall@3
- type: dot_recall@5
value: 0.117
name: Dot Recall@5
- type: dot_recall@10
value: 0.18866666666666668
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.1725314970247529
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.2993571428571428
name: Dot Mrr@10
- type: dot_map@100
value: 0.1316070110116111
name: Dot Map@100
- type: query_active_dims
value: 390.1000061035156
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.98721905490782
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 708.0321655273438
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9768025632158004
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoArguAna
type: NanoArguAna
metrics:
- type: dot_accuracy@1
value: 0.06
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.14
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.16
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.18
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.06
name: Dot Precision@1
- type: dot_precision@3
value: 0.04666666666666667
name: Dot Precision@3
- type: dot_precision@5
value: 0.032
name: Dot Precision@5
- type: dot_precision@10
value: 0.018000000000000002
name: Dot Precision@10
- type: dot_recall@1
value: 0.06
name: Dot Recall@1
- type: dot_recall@3
value: 0.14
name: Dot Recall@3
- type: dot_recall@5
value: 0.16
name: Dot Recall@5
- type: dot_recall@10
value: 0.18
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.11875554281390689
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.09933333333333332
name: Dot Mrr@10
- type: dot_map@100
value: 0.10937808986204274
name: Dot Map@100
- type: query_active_dims
value: 699.3400268554688
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9770873459519209
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 637.1741333007812
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9791241028339958
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoSciFact
type: NanoSciFact
metrics:
- type: dot_accuracy@1
value: 0.28
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.38
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.48
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.56
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.28
name: Dot Precision@1
- type: dot_precision@3
value: 0.13333333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.10400000000000001
name: Dot Precision@5
- type: dot_precision@10
value: 0.066
name: Dot Precision@10
- type: dot_recall@1
value: 0.245
name: Dot Recall@1
- type: dot_recall@3
value: 0.345
name: Dot Recall@3
- type: dot_recall@5
value: 0.455
name: Dot Recall@5
- type: dot_recall@10
value: 0.55
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.39370005539130326
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.3605238095238095
name: Dot Mrr@10
- type: dot_map@100
value: 0.34373433911333495
name: Dot Map@100
- type: query_active_dims
value: 591.9000244140625
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9806074299058364
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 700.0397338867188
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9770644212736151
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoTouche2020
type: NanoTouche2020
metrics:
- type: dot_accuracy@1
value: 0.6326530612244898
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.8775510204081632
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.9387755102040817
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 1
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.6326530612244898
name: Dot Precision@1
- type: dot_precision@3
value: 0.6054421768707483
name: Dot Precision@3
- type: dot_precision@5
value: 0.5428571428571428
name: Dot Precision@5
- type: dot_precision@10
value: 0.4448979591836735
name: Dot Precision@10
- type: dot_recall@1
value: 0.04381728832224134
name: Dot Recall@1
- type: dot_recall@3
value: 0.12154307114357087
name: Dot Recall@3
- type: dot_recall@5
value: 0.18182574968079382
name: Dot Recall@5
- type: dot_recall@10
value: 0.28743001839116805
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5082094202851214
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.768415937803693
name: Dot Mrr@10
- type: dot_map@100
value: 0.37730310641309606
name: Dot Map@100
- type: query_active_dims
value: 37.14285659790039
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.998783079201956
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 505.10235595703125
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9834512038543664
name: Corpus Sparsity Ratio
bert-base-uncased adapter finetuned on GooAQ pairs
This is a SPLADE Sparse Encoder model finetuned from google-bert/bert-base-uncased on the gooaq dataset using the sentence-transformers library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
Model Details
Model Description
- Model Type: SPLADE Sparse Encoder
- Base model: google-bert/bert-base-uncased
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 30522 dimensions
- Similarity Function: Dot Product
- Training Dataset:
- Language: en
- License: apache-2.0
Model Sources
- Documentation: Sentence Transformers Documentation
- Documentation: Sparse Encoder Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Sparse Encoders on Hugging Face
Full Model Architecture
SparseEncoder(
(0): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertForMaskedLM'})
(1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SparseEncoder
# Download from the 🤗 Hub
model = SparseEncoder("tomaarsen/splade-bert-base-uncased-gooaq-peft")
# Run inference
queries = [
"how many days for doxycycline to work on sinus infection?",
]
documents = [
'Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks for chronic sinusitis.',
'Most engagements typically have a cocktail dress code, calling for dresses at, or slightly above, knee-length and high heels. If your party states a different dress code, however, such as semi-formal or dressy-casual, you may need to dress up or down accordingly.',
'The average service life of a gas furnace is about 15 years, but the actual life span of an individual unit can vary greatly. There are a number of contributing factors that determine the age a furnace reaches: The quality of the equipment.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 30522] [3, 30522]
# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[90.9602, 31.2714, 39.4626]])
Evaluation
Metrics
Sparse Information Retrieval
- Datasets:
NanoMSMARCO
,NanoNFCorpus
,NanoNQ
,NanoClimateFEVER
,NanoDBPedia
,NanoFEVER
,NanoFiQA2018
,NanoHotpotQA
,NanoMSMARCO
,NanoNFCorpus
,NanoNQ
,NanoQuoraRetrieval
,NanoSCIDOCS
,NanoArguAna
,NanoSciFact
andNanoTouche2020
- Evaluated with
SparseInformationRetrievalEvaluator
Metric | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
dot_accuracy@1 | 0.18 | 0.28 | 0.26 | 0.18 | 0.54 | 0.34 | 0.16 | 0.64 | 0.2 | 0.18 | 0.06 | 0.28 | 0.6327 |
dot_accuracy@3 | 0.42 | 0.4 | 0.46 | 0.32 | 0.7 | 0.56 | 0.28 | 0.78 | 0.36 | 0.38 | 0.14 | 0.38 | 0.8776 |
dot_accuracy@5 | 0.5 | 0.5 | 0.62 | 0.36 | 0.84 | 0.58 | 0.36 | 0.84 | 0.42 | 0.42 | 0.16 | 0.48 | 0.9388 |
dot_accuracy@10 | 0.66 | 0.52 | 0.72 | 0.5 | 0.88 | 0.7 | 0.46 | 0.96 | 0.48 | 0.6 | 0.18 | 0.56 | 1.0 |
dot_precision@1 | 0.18 | 0.28 | 0.26 | 0.18 | 0.54 | 0.34 | 0.16 | 0.64 | 0.2 | 0.18 | 0.06 | 0.28 | 0.6327 |
dot_precision@3 | 0.14 | 0.2133 | 0.1533 | 0.12 | 0.4533 | 0.1933 | 0.12 | 0.32 | 0.1333 | 0.1667 | 0.0467 | 0.1333 | 0.6054 |
dot_precision@5 | 0.1 | 0.204 | 0.124 | 0.092 | 0.452 | 0.12 | 0.112 | 0.228 | 0.092 | 0.116 | 0.032 | 0.104 | 0.5429 |
dot_precision@10 | 0.066 | 0.192 | 0.074 | 0.07 | 0.414 | 0.072 | 0.076 | 0.136 | 0.052 | 0.092 | 0.018 | 0.066 | 0.4449 |
dot_recall@1 | 0.18 | 0.0096 | 0.25 | 0.0967 | 0.045 | 0.32 | 0.0666 | 0.32 | 0.1767 | 0.036 | 0.06 | 0.245 | 0.0438 |
dot_recall@3 | 0.42 | 0.0207 | 0.43 | 0.159 | 0.0985 | 0.55 | 0.1672 | 0.48 | 0.3273 | 0.101 | 0.14 | 0.345 | 0.1215 |
dot_recall@5 | 0.5 | 0.0334 | 0.58 | 0.1907 | 0.1481 | 0.57 | 0.2236 | 0.57 | 0.3873 | 0.117 | 0.16 | 0.455 | 0.1818 |
dot_recall@10 | 0.66 | 0.0577 | 0.68 | 0.2657 | 0.2518 | 0.6767 | 0.3206 | 0.68 | 0.434 | 0.1887 | 0.18 | 0.55 | 0.2874 |
dot_ndcg@10 | 0.4047 | 0.2049 | 0.4524 | 0.2106 | 0.4748 | 0.5066 | 0.227 | 0.5959 | 0.3167 | 0.1725 | 0.1188 | 0.3937 | 0.5082 |
dot_mrr@10 | 0.3246 | 0.3592 | 0.3947 | 0.2611 | 0.6571 | 0.4605 | 0.2492 | 0.7347 | 0.2916 | 0.2994 | 0.0993 | 0.3605 | 0.7684 |
dot_map@100 | 0.3399 | 0.0703 | 0.3832 | 0.1654 | 0.3339 | 0.46 | 0.1797 | 0.5052 | 0.2947 | 0.1316 | 0.1094 | 0.3437 | 0.3773 |
query_active_dims | 119.92 | 204.08 | 104.48 | 464.26 | 190.66 | 396.88 | 137.84 | 147.22 | 77.0 | 390.1 | 699.34 | 591.9 | 37.1429 |
query_sparsity_ratio | 0.9961 | 0.9933 | 0.9966 | 0.9848 | 0.9938 | 0.987 | 0.9955 | 0.9952 | 0.9975 | 0.9872 | 0.9771 | 0.9806 | 0.9988 |
corpus_active_dims | 558.7083 | 768.1585 | 660.2721 | 703.1382 | 732.0422 | 838.1882 | 472.5312 | 793.5137 | 72.0059 | 708.0322 | 637.1741 | 700.0397 | 505.1024 |
corpus_sparsity_ratio | 0.9817 | 0.9748 | 0.9784 | 0.977 | 0.976 | 0.9725 | 0.9845 | 0.974 | 0.9976 | 0.9768 | 0.9791 | 0.9771 | 0.9835 |
Sparse Nano BEIR
- Dataset:
NanoBEIR_mean
- Evaluated with
SparseNanoBEIREvaluator
with these parameters:{ "dataset_names": [ "msmarco", "nfcorpus", "nq" ] }
Metric | Value |
---|---|
dot_accuracy@1 | 0.24 |
dot_accuracy@3 | 0.4333 |
dot_accuracy@5 | 0.54 |
dot_accuracy@10 | 0.6133 |
dot_precision@1 | 0.24 |
dot_precision@3 | 0.1733 |
dot_precision@5 | 0.1453 |
dot_precision@10 | 0.1067 |
dot_recall@1 | 0.1465 |
dot_recall@3 | 0.3004 |
dot_recall@5 | 0.3778 |
dot_recall@10 | 0.4335 |
dot_ndcg@10 | 0.3478 |
dot_mrr@10 | 0.359 |
dot_map@100 | 0.2683 |
query_active_dims | 158.4267 |
query_sparsity_ratio | 0.9948 |
corpus_active_dims | 649.7461 |
corpus_sparsity_ratio | 0.9787 |
Sparse Nano BEIR
- Dataset:
NanoBEIR_mean
- Evaluated with
SparseNanoBEIREvaluator
with these parameters:{ "dataset_names": [ "climatefever", "dbpedia", "fever", "fiqa2018", "hotpotqa", "msmarco", "nfcorpus", "nq", "quoraretrieval", "scidocs", "arguana", "scifact", "touche2020" ] }
Metric | Value |
---|---|
dot_accuracy@1 | 0.3025 |
dot_accuracy@3 | 0.466 |
dot_accuracy@5 | 0.5399 |
dot_accuracy@10 | 0.6323 |
dot_precision@1 | 0.3025 |
dot_precision@3 | 0.2153 |
dot_precision@5 | 0.1784 |
dot_precision@10 | 0.1364 |
dot_recall@1 | 0.1423 |
dot_recall@3 | 0.2585 |
dot_recall@5 | 0.3167 |
dot_recall@10 | 0.4025 |
dot_ndcg@10 | 0.3528 |
dot_mrr@10 | 0.4046 |
dot_map@100 | 0.2842 |
query_active_dims | 274.2743 |
query_sparsity_ratio | 0.991 |
corpus_active_dims | 613.3642 |
corpus_sparsity_ratio | 0.9799 |
Training Details
Training Dataset
gooaq
- Dataset: gooaq at b089f72
- Size: 99,000 training samples
- Columns:
question
andanswer
- Approximate statistics based on the first 1000 samples:
question answer type string string details - min: 8 tokens
- mean: 11.79 tokens
- max: 24 tokens
- min: 14 tokens
- mean: 60.02 tokens
- max: 153 tokens
- Samples:
question answer what are the 5 characteristics of a star?
Key Concept: Characteristics used to classify stars include color, temperature, size, composition, and brightness.
are copic markers alcohol ink?
Copic Ink is alcohol-based and flammable. Keep away from direct sunlight and extreme temperatures.
what is the difference between appellate term and appellate division?
Appellate terms An appellate term is an intermediate appellate court that hears appeals from the inferior courts within their designated counties or judicial districts, and are intended to ease the workload on the Appellate Division and provide a less expensive forum closer to the people.
- Loss:
SpladeLoss
with these parameters:{ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')", "document_regularizer_weight": 3e-05, "query_regularizer_weight": 5e-05 }
Evaluation Dataset
gooaq
- Dataset: gooaq at b089f72
- Size: 1,000 evaluation samples
- Columns:
question
andanswer
- Approximate statistics based on the first 1000 samples:
question answer type string string details - min: 8 tokens
- mean: 11.93 tokens
- max: 25 tokens
- min: 14 tokens
- mean: 60.84 tokens
- max: 127 tokens
- Samples:
question answer should you take ibuprofen with high blood pressure?
In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor.
how old do you have to be to work in sc?
The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor.
how to write a topic proposal for a research paper?
['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.']
- Loss:
SpladeLoss
with these parameters:{ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')", "document_regularizer_weight": 3e-05, "query_regularizer_weight": 5e-05 }
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy
: stepsper_device_train_batch_size
: 32per_device_eval_batch_size
: 32learning_rate
: 2e-05num_train_epochs
: 1bf16
: Trueload_best_model_at_end
: Truebatch_sampler
: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir
: Falsedo_predict
: Falseeval_strategy
: stepsprediction_loss_only
: Trueper_device_train_batch_size
: 32per_device_eval_batch_size
: 32per_gpu_train_batch_size
: Noneper_gpu_eval_batch_size
: Nonegradient_accumulation_steps
: 1eval_accumulation_steps
: Nonetorch_empty_cache_steps
: Nonelearning_rate
: 2e-05weight_decay
: 0.0adam_beta1
: 0.9adam_beta2
: 0.999adam_epsilon
: 1e-08max_grad_norm
: 1.0num_train_epochs
: 1max_steps
: -1lr_scheduler_type
: linearlr_scheduler_kwargs
: {}warmup_ratio
: 0.0warmup_steps
: 0log_level
: passivelog_level_replica
: warninglog_on_each_node
: Truelogging_nan_inf_filter
: Truesave_safetensors
: Truesave_on_each_node
: Falsesave_only_model
: Falserestore_callback_states_from_checkpoint
: Falseno_cuda
: Falseuse_cpu
: Falseuse_mps_device
: Falseseed
: 42data_seed
: Nonejit_mode_eval
: Falseuse_ipex
: Falsebf16
: Truefp16
: Falsefp16_opt_level
: O1half_precision_backend
: autobf16_full_eval
: Falsefp16_full_eval
: Falsetf32
: Nonelocal_rank
: 0ddp_backend
: Nonetpu_num_cores
: Nonetpu_metrics_debug
: Falsedebug
: []dataloader_drop_last
: Falsedataloader_num_workers
: 0dataloader_prefetch_factor
: Nonepast_index
: -1disable_tqdm
: Falseremove_unused_columns
: Truelabel_names
: Noneload_best_model_at_end
: Trueignore_data_skip
: Falsefsdp
: []fsdp_min_num_params
: 0fsdp_config
: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap
: Noneaccelerator_config
: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed
: Nonelabel_smoothing_factor
: 0.0optim
: adamw_torchoptim_args
: Noneadafactor
: Falsegroup_by_length
: Falselength_column_name
: lengthddp_find_unused_parameters
: Noneddp_bucket_cap_mb
: Noneddp_broadcast_buffers
: Falsedataloader_pin_memory
: Truedataloader_persistent_workers
: Falseskip_memory_metrics
: Trueuse_legacy_prediction_loop
: Falsepush_to_hub
: Falseresume_from_checkpoint
: Nonehub_model_id
: Nonehub_strategy
: every_savehub_private_repo
: Nonehub_always_push
: Falsegradient_checkpointing
: Falsegradient_checkpointing_kwargs
: Noneinclude_inputs_for_metrics
: Falseinclude_for_metrics
: []eval_do_concat_batches
: Truefp16_backend
: autopush_to_hub_model_id
: Nonepush_to_hub_organization
: Nonemp_parameters
:auto_find_batch_size
: Falsefull_determinism
: Falsetorchdynamo
: Noneray_scope
: lastddp_timeout
: 1800torch_compile
: Falsetorch_compile_backend
: Nonetorch_compile_mode
: Noneinclude_tokens_per_second
: Falseinclude_num_input_tokens_seen
: Falseneftune_noise_alpha
: Noneoptim_target_modules
: Nonebatch_eval_metrics
: Falseeval_on_start
: Falseuse_liger_kernel
: Falseeval_use_gather_object
: Falseaverage_tokens_across_devices
: Falseprompts
: Nonebatch_sampler
: no_duplicatesmulti_dataset_batch_sampler
: proportionalrouter_mapping
: {}learning_rate_mapping
: {}
Training Logs
Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0.0323 | 100 | 91.8917 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.0646 | 200 | 8.1616 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.0970 | 300 | 1.6108 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.1293 | 400 | 0.7834 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.1616 | 500 | 0.5234 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.1939 | 600 | 0.396 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.1972 | 610 | - | 0.3011 | 0.3418 | 0.1789 | 0.2676 | 0.2628 | - | - | - | - | - | - | - | - | - | - |
0.2262 | 700 | 0.4065 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.2586 | 800 | 0.3367 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.2909 | 900 | 0.3305 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.3232 | 1000 | 0.3454 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.3555 | 1100 | 0.284 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.3878 | 1200 | 0.2639 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.3943 | 1220 | - | 0.2610 | 0.4244 | 0.2060 | 0.3846 | 0.3383 | - | - | - | - | - | - | - | - | - | - |
0.4202 | 1300 | 0.2787 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.4525 | 1400 | 0.2343 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.4848 | 1500 | 0.2833 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.5171 | 1600 | 0.2706 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.5495 | 1700 | 0.2277 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.5818 | 1800 | 0.2515 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.5915 | 1830 | - | 0.2282 | 0.3836 | 0.1977 | 0.3801 | 0.3204 | - | - | - | - | - | - | - | - | - | - |
0.6141 | 1900 | 0.1985 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.6464 | 2000 | 0.2224 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.6787 | 2100 | 0.1904 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.7111 | 2200 | 0.2564 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.7434 | 2300 | 0.2333 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.7757 | 2400 | 0.208 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.7886 | 2440 | - | 0.2087 | 0.4047 | 0.2049 | 0.4524 | 0.354 | - | - | - | - | - | - | - | - | - | - |
0.8080 | 2500 | 0.1874 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.8403 | 2600 | 0.2296 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.8727 | 2700 | 0.1926 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.9050 | 2800 | 0.2023 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.9373 | 2900 | 0.2098 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.9696 | 3000 | 0.1607 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
0.9858 | 3050 | - | 0.2088 | 0.3842 | 0.2030 | 0.4562 | 0.3478 | - | - | - | - | - | - | - | - | - | - |
-1 | -1 | - | - | 0.4047 | 0.2049 | 0.4524 | 0.3528 | 0.2106 | 0.4748 | 0.5066 | 0.2270 | 0.5959 | 0.3167 | 0.1725 | 0.1188 | 0.3937 | 0.5082 |
- The bold row denotes the saved checkpoint.
Environmental Impact
Carbon emissions were measured using CodeCarbon.
- Energy Consumed: 0.058 kWh
- Carbon Emitted: 0.023 kg of CO2
- Hours Used: 0.232 hours
Training Hardware
- On Cloud: No
- GPU Model: 1 x NVIDIA GeForce RTX 3090
- CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
- RAM Size: 31.78 GB
Framework Versions
- Python: 3.11.6
- Sentence Transformers: 4.2.0.dev0
- Transformers: 4.52.4
- PyTorch: 2.7.1+cu126
- Accelerate: 1.5.1
- Datasets: 2.21.0
- Tokenizers: 0.21.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
SpladeLoss
@misc{formal2022distillationhardnegativesampling,
title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
year={2022},
eprint={2205.04733},
archivePrefix={arXiv},
primaryClass={cs.IR},
url={https://arxiv.org/abs/2205.04733},
}
SparseMultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
FlopsLoss
@article{paria2020minimizing,
title={Minimizing flops to learn efficient sparse representations},
author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s},
journal={arXiv preprint arXiv:2004.05665},
year={2020}
}