Hugging Face's logo

redis
/
langcache-embed-v3

Sentence Similarity
sentence-transformers
Safetensors
English
modernbert
biencoder
text-classification
sentence-pair-classification
semantic-similarity
semantic-search
retrieval
reranking
Generated from Trainer
dataset_size:9233417
loss:ArcFaceInBatchLoss
Eval Results
text-embeddings-inference
Model card Files
xet
Community
radoslavralev commited on
Commit
9fcad6e
·
verified ·
1 Parent(s): 465c34b

Add new SentenceTransformer model

Browse files
Files changed (2) hide show
  1. README.md +364 -18
  2. model.safetensors +1 -1
README.md CHANGED
@@ -79,22 +79,22 @@ model-index:
79
  type: test
80
  metrics:
81
  - type: cosine_accuracy@1
82
- value: 0.4134460403309574
83
  name: Cosine Accuracy@1
84
  - type: cosine_precision@1
85
- value: 0.4134460403309574
86
  name: Cosine Precision@1
87
  - type: cosine_recall@1
88
- value: 0.39978666009460445
89
  name: Cosine Recall@1
90
  - type: cosine_ndcg@10
91
- value: 0.5951872317507402
92
  name: Cosine Ndcg@10
93
  - type: cosine_mrr@1
94
- value: 0.4134460403309574
95
  name: Cosine Mrr@1
96
  - type: cosine_map@100
97
- value: 0.5433812918329537
98
  name: Cosine Map@100
99
  ---
100
 
@@ -159,9 +159,9 @@ print(embeddings.shape)
159
  # Get the similarity scores for the embeddings
160
  similarities = model.similarity(embeddings, embeddings)
161
  print(similarities)
162
- # tensor([[0.9961, 0.9922, 0.9922],
163
- # [0.9922, 1.0000, 1.0000],
164
- # [0.9922, 1.0000, 1.0000]], dtype=torch.bfloat16)
165
  ```
166
 
167
  <!--
@@ -199,12 +199,12 @@ You can finetune this model on your own dataset.
199
 
200
  | Metric | Value |
201
  |:-------------------|:-----------|
202
- | cosine_accuracy@1 | 0.4134 |
203
- | cosine_precision@1 | 0.4134 |
204
- | cosine_recall@1 | 0.3998 |
205
- | **cosine_ndcg@10** | **0.5952** |
206
- | cosine_mrr@1 | 0.4134 |
207
- | cosine_map@100 | 0.5434 |
208
 
209
  <!--
210
  ## Bias, Risks and Limitations
@@ -274,11 +274,357 @@ You can finetune this model on your own dataset.
274
  }
275
  ```
276
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
277
  ### Training Logs
278
- | Epoch | Step | test_cosine_ndcg@10 |
279
- |:-----:|:----:|:-------------------:|
280
- | -1 | -1 | 0.5952 |
281
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
282
 
283
  ### Framework Versions
284
  - Python: 3.12.3
 
79
  type: test
80
  metrics:
81
  - type: cosine_accuracy@1
82
+ value: 0.6032809198037179
83
  name: Cosine Accuracy@1
84
  - type: cosine_precision@1
85
+ value: 0.6032809198037179
86
  name: Cosine Precision@1
87
  - type: cosine_recall@1
88
+ value: 0.585771482488324
89
  name: Cosine Recall@1
90
  - type: cosine_ndcg@10
91
+ value: 0.7747479314468421
92
  name: Cosine Ndcg@10
93
  - type: cosine_mrr@1
94
+ value: 0.6032809198037179
95
  name: Cosine Mrr@1
96
  - type: cosine_map@100
97
+ value: 0.7280398908979986
98
  name: Cosine Map@100
99
  ---
100
 
 
159
  # Get the similarity scores for the embeddings
160
  similarities = model.similarity(embeddings, embeddings)
161
  print(similarities)
162
+ # tensor([[1.0000, 0.9180, 0.4531],
163
+ # [0.9180, 1.0000, 0.4746],
164
+ # [0.4531, 0.4746, 1.0000]], dtype=torch.bfloat16)
165
  ```
166
 
167
  <!--
 
199
 
200
  | Metric | Value |
201
  |:-------------------|:-----------|
202
+ | cosine_accuracy@1 | 0.6033 |
203
+ | cosine_precision@1 | 0.6033 |
204
+ | cosine_recall@1 | 0.5858 |
205
+ | **cosine_ndcg@10** | **0.7747** |
206
+ | cosine_mrr@1 | 0.6033 |
207
+ | cosine_map@100 | 0.728 |
208
 
209
  <!--
210
  ## Bias, Risks and Limitations
 
274
  }
275
  ```
276
 
277
+ ### Training Hyperparameters
278
+ #### Non-Default Hyperparameters
279
+
280
+ - `eval_strategy`: steps
281
+ - `per_device_train_batch_size`: 128
282
+ - `per_device_eval_batch_size`: 128
283
+ - `weight_decay`: 0.001
284
+ - `adam_beta2`: 0.98
285
+ - `adam_epsilon`: 1e-06
286
+ - `max_steps`: 100000
287
+ - `warmup_ratio`: 0.1
288
+ - `load_best_model_at_end`: True
289
+ - `optim`: stable_adamw
290
+ - `ddp_find_unused_parameters`: False
291
+ - `push_to_hub`: True
292
+ - `hub_model_id`: redis/langcache-embed-v3
293
+ - `batch_sampler`: no_duplicates
294
+
295
+ #### All Hyperparameters
296
+ <details><summary>Click to expand</summary>
297
+
298
+ - `overwrite_output_dir`: False
299
+ - `do_predict`: False
300
+ - `eval_strategy`: steps
301
+ - `prediction_loss_only`: True
302
+ - `per_device_train_batch_size`: 128
303
+ - `per_device_eval_batch_size`: 128
304
+ - `per_gpu_train_batch_size`: None
305
+ - `per_gpu_eval_batch_size`: None
306
+ - `gradient_accumulation_steps`: 1
307
+ - `eval_accumulation_steps`: None
308
+ - `torch_empty_cache_steps`: None
309
+ - `learning_rate`: 5e-05
310
+ - `weight_decay`: 0.001
311
+ - `adam_beta1`: 0.9
312
+ - `adam_beta2`: 0.98
313
+ - `adam_epsilon`: 1e-06
314
+ - `max_grad_norm`: 1.0
315
+ - `num_train_epochs`: 3.0
316
+ - `max_steps`: 100000
317
+ - `lr_scheduler_type`: linear
318
+ - `lr_scheduler_kwargs`: {}
319
+ - `warmup_ratio`: 0.1
320
+ - `warmup_steps`: 0
321
+ - `log_level`: passive
322
+ - `log_level_replica`: warning
323
+ - `log_on_each_node`: True
324
+ - `logging_nan_inf_filter`: True
325
+ - `save_safetensors`: True
326
+ - `save_on_each_node`: False
327
+ - `save_only_model`: False
328
+ - `restore_callback_states_from_checkpoint`: False
329
+ - `no_cuda`: False
330
+ - `use_cpu`: False
331
+ - `use_mps_device`: False
332
+ - `seed`: 42
333
+ - `data_seed`: None
334
+ - `jit_mode_eval`: False
335
+ - `use_ipex`: False
336
+ - `bf16`: False
337
+ - `fp16`: False
338
+ - `fp16_opt_level`: O1
339
+ - `half_precision_backend`: auto
340
+ - `bf16_full_eval`: False
341
+ - `fp16_full_eval`: False
342
+ - `tf32`: None
343
+ - `local_rank`: 0
344
+ - `ddp_backend`: None
345
+ - `tpu_num_cores`: None
346
+ - `tpu_metrics_debug`: False
347
+ - `debug`: []
348
+ - `dataloader_drop_last`: False
349
+ - `dataloader_num_workers`: 0
350
+ - `dataloader_prefetch_factor`: None
351
+ - `past_index`: -1
352
+ - `disable_tqdm`: False
353
+ - `remove_unused_columns`: True
354
+ - `label_names`: None
355
+ - `load_best_model_at_end`: True
356
+ - `ignore_data_skip`: False
357
+ - `fsdp`: []
358
+ - `fsdp_min_num_params`: 0
359
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
360
+ - `fsdp_transformer_layer_cls_to_wrap`: None
361
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
362
+ - `parallelism_config`: None
363
+ - `deepspeed`: None
364
+ - `label_smoothing_factor`: 0.0
365
+ - `optim`: stable_adamw
366
+ - `optim_args`: None
367
+ - `adafactor`: False
368
+ - `group_by_length`: False
369
+ - `length_column_name`: length
370
+ - `ddp_find_unused_parameters`: False
371
+ - `ddp_bucket_cap_mb`: None
372
+ - `ddp_broadcast_buffers`: False
373
+ - `dataloader_pin_memory`: True
374
+ - `dataloader_persistent_workers`: False
375
+ - `skip_memory_metrics`: True
376
+ - `use_legacy_prediction_loop`: False
377
+ - `push_to_hub`: True
378
+ - `resume_from_checkpoint`: None
379
+ - `hub_model_id`: redis/langcache-embed-v3
380
+ - `hub_strategy`: every_save
381
+ - `hub_private_repo`: None
382
+ - `hub_always_push`: False
383
+ - `hub_revision`: None
384
+ - `gradient_checkpointing`: False
385
+ - `gradient_checkpointing_kwargs`: None
386
+ - `include_inputs_for_metrics`: False
387
+ - `include_for_metrics`: []
388
+ - `eval_do_concat_batches`: True
389
+ - `fp16_backend`: auto
390
+ - `push_to_hub_model_id`: None
391
+ - `push_to_hub_organization`: None
392
+ - `mp_parameters`:
393
+ - `auto_find_batch_size`: False
394
+ - `full_determinism`: False
395
+ - `torchdynamo`: None
396
+ - `ray_scope`: last
397
+ - `ddp_timeout`: 1800
398
+ - `torch_compile`: False
399
+ - `torch_compile_backend`: None
400
+ - `torch_compile_mode`: None
401
+ - `include_tokens_per_second`: False
402
+ - `include_num_input_tokens_seen`: False
403
+ - `neftune_noise_alpha`: None
404
+ - `optim_target_modules`: None
405
+ - `batch_eval_metrics`: False
406
+ - `eval_on_start`: False
407
+ - `use_liger_kernel`: False
408
+ - `liger_kernel_config`: None
409
+ - `eval_use_gather_object`: False
410
+ - `average_tokens_across_devices`: False
411
+ - `prompts`: None
412
+ - `batch_sampler`: no_duplicates
413
+ - `multi_dataset_batch_sampler`: proportional
414
+ - `router_mapping`: {}
415
+ - `learning_rate_mapping`: {}
416
+
417
+ </details>
418
+
419
  ### Training Logs
420
+ <details><summary>Click to expand</summary>
 
 
421
 
422
+ | Epoch | Step | Training Loss | Validation Loss | test_cosine_ndcg@10 |
423
+ |:----------:|:---------:|:-------------:|:---------------:|:-------------------:|
424
+ | -1 | -1 | - | - | 0.5952 |
425
+ | 0.0069 | 500 | 3.4812 | 0.6932 | 0.6810 |
426
+ | 0.0139 | 1000 | 0.6045 | 0.4804 | 0.7354 |
427
+ | 0.0208 | 1500 | 0.3127 | 0.4128 | 0.7437 |
428
+ | 0.0277 | 2000 | 0.2424 | 0.4077 | 0.7440 |
429
+ | 0.0347 | 2500 | 0.2027 | 0.3707 | 0.7501 |
430
+ | 0.0416 | 3000 | 0.1752 | 0.3453 | 0.7551 |
431
+ | 0.0485 | 3500 | 0.1622 | 0.3380 | 0.7540 |
432
+ | 0.0555 | 4000 | 0.1466 | 0.3185 | 0.7583 |
433
+ | 0.0624 | 4500 | 0.1392 | 0.3092 | 0.7588 |
434
+ | 0.0693 | 5000 | 0.1342 | 0.3054 | 0.7566 |
435
+ | 0.0762 | 5500 | 0.1291 | 0.2960 | 0.7582 |
436
+ | 0.0832 | 6000 | 0.1291 | 0.2856 | 0.7616 |
437
+ | 0.0901 | 6500 | 0.1199 | 0.2803 | 0.7624 |
438
+ | 0.0970 | 7000 | 0.1171 | 0.2692 | 0.7648 |
439
+ | 0.1040 | 7500 | 0.1097 | 0.2811 | 0.7629 |
440
+ | 0.1109 | 8000 | 0.1089 | 0.2901 | 0.7621 |
441
+ | 0.1178 | 8500 | 0.1088 | 0.2986 | 0.7568 |
442
+ | 0.1248 | 9000 | 0.109 | 0.2806 | 0.7628 |
443
+ | 0.1317 | 9500 | 0.1046 | 0.3050 | 0.7587 |
444
+ | 0.1386 | 10000 | 0.1035 | 0.2925 | 0.7596 |
445
+ | 0.1456 | 10500 | 0.1041 | 0.2940 | 0.7573 |
446
+ | 0.1525 | 11000 | 0.1023 | 0.2790 | 0.7632 |
447
+ | 0.1594 | 11500 | 0.0992 | 0.3293 | 0.7542 |
448
+ | 0.1664 | 12000 | 0.0996 | 0.2876 | 0.7570 |
449
+ | 0.1733 | 12500 | 0.0949 | 0.2881 | 0.7591 |
450
+ | 0.1802 | 13000 | 0.0921 | 0.2861 | 0.7598 |
451
+ | 0.1871 | 13500 | 0.0912 | 0.2763 | 0.7632 |
452
+ | 0.1941 | 14000 | 0.0912 | 0.2785 | 0.7643 |
453
+ | 0.2010 | 14500 | 0.0909 | 0.3198 | 0.7629 |
454
+ | 0.2079 | 15000 | 0.0911 | 0.3015 | 0.7575 |
455
+ | 0.2149 | 15500 | 0.0861 | 0.3029 | 0.7597 |
456
+ | 0.2218 | 16000 | 0.0857 | 0.3271 | 0.7568 |
457
+ | 0.2287 | 16500 | 0.0843 | 0.2579 | 0.7645 |
458
+ | 0.2357 | 17000 | 0.085 | 0.2923 | 0.7625 |
459
+ | 0.2426 | 17500 | 0.0846 | 0.3241 | 0.7598 |
460
+ | 0.2495 | 18000 | 0.083 | 0.3128 | 0.7616 |
461
+ | 0.2565 | 18500 | 0.0794 | 0.2926 | 0.7611 |
462
+ | 0.2634 | 19000 | 0.0806 | 0.2665 | 0.7640 |
463
+ | 0.2703 | 19500 | 0.0782 | 0.2963 | 0.7615 |
464
+ | 0.2773 | 20000 | 0.0786 | 0.2771 | 0.7611 |
465
+ | 0.2842 | 20500 | 0.0761 | 0.2853 | 0.7623 |
466
+ | 0.2911 | 21000 | 0.0752 | 0.2782 | 0.7626 |
467
+ | 0.2980 | 21500 | 0.0777 | 0.2680 | 0.7612 |
468
+ | 0.3050 | 22000 | 0.0782 | 0.2731 | 0.7636 |
469
+ | 0.3119 | 22500 | 0.0785 | 0.2627 | 0.7627 |
470
+ | 0.3188 | 23000 | 0.0741 | 0.2714 | 0.7613 |
471
+ | 0.3258 | 23500 | 0.0741 | 0.2713 | 0.7661 |
472
+ | 0.3327 | 24000 | 0.072 | 0.2630 | 0.7636 |
473
+ | 0.3396 | 24500 | 0.0739 | 0.2839 | 0.7648 |
474
+ | 0.3466 | 25000 | 0.07 | 0.2860 | 0.7634 |
475
+ | 0.3535 | 25500 | 0.0715 | 0.2612 | 0.7666 |
476
+ | 0.3604 | 26000 | 0.0711 | 0.2531 | 0.7671 |
477
+ | 0.3674 | 26500 | 0.0701 | 0.2682 | 0.7638 |
478
+ | 0.3743 | 27000 | 0.0733 | 0.2708 | 0.7635 |
479
+ | 0.3812 | 27500 | 0.0705 | 0.2873 | 0.7636 |
480
+ | 0.3882 | 28000 | 0.0663 | 0.2831 | 0.7647 |
481
+ | 0.3951 | 28500 | 0.0678 | 0.2825 | 0.7643 |
482
+ | 0.4020 | 29000 | 0.0691 | 0.2733 | 0.7654 |
483
+ | 0.4089 | 29500 | 0.0696 | 0.2831 | 0.7621 |
484
+ | 0.4159 | 30000 | 0.0708 | 0.2893 | 0.7643 |
485
+ | 0.4228 | 30500 | 0.0663 | 0.2758 | 0.7653 |
486
+ | 0.4297 | 31000 | 0.064 | 0.2589 | 0.7666 |
487
+ | 0.4367 | 31500 | 0.0636 | 0.2491 | 0.7681 |
488
+ | 0.4436 | 32000 | 0.0644 | 0.2601 | 0.7650 |
489
+ | 0.4505 | 32500 | 0.0655 | 0.2611 | 0.7668 |
490
+ | 0.4575 | 33000 | 0.0643 | 0.2597 | 0.7664 |
491
+ | 0.4644 | 33500 | 0.066 | 0.2696 | 0.7677 |
492
+ | 0.4713 | 34000 | 0.0664 | 0.2489 | 0.7690 |
493
+ | 0.4783 | 34500 | 0.0654 | 0.2644 | 0.7649 |
494
+ | 0.4852 | 35000 | 0.0653 | 0.2704 | 0.7665 |
495
+ | 0.4921 | 35500 | 0.0657 | 0.2578 | 0.7689 |
496
+ | 0.4991 | 36000 | 0.0634 | 0.2629 | 0.7669 |
497
+ | 0.5060 | 36500 | 0.0609 | 0.2631 | 0.7663 |
498
+ | 0.5129 | 37000 | 0.0646 | 0.2586 | 0.7667 |
499
+ | 0.5198 | 37500 | 0.0634 | 0.2572 | 0.7657 |
500
+ | 0.5268 | 38000 | 0.0607 | 0.2624 | 0.7664 |
501
+ | 0.5337 | 38500 | 0.0621 | 0.2622 | 0.7668 |
502
+ | 0.5406 | 39000 | 0.0614 | 0.2562 | 0.7676 |
503
+ | 0.5476 | 39500 | 0.0621 | 0.2560 | 0.7673 |
504
+ | 0.5545 | 40000 | 0.0608 | 0.2506 | 0.7684 |
505
+ | 0.5614 | 40500 | 0.0621 | 0.2718 | 0.7666 |
506
+ | 0.5684 | 41000 | 0.0598 | 0.2599 | 0.7700 |
507
+ | 0.5753 | 41500 | 0.06 | 0.2706 | 0.7687 |
508
+ | 0.5822 | 42000 | 0.0618 | 0.2635 | 0.7694 |
509
+ | 0.5892 | 42500 | 0.0604 | 0.2743 | 0.7660 |
510
+ | 0.5961 | 43000 | 0.0576 | 0.2733 | 0.7661 |
511
+ | 0.6030 | 43500 | 0.0597 | 0.2644 | 0.7712 |
512
+ | 0.6100 | 44000 | 0.0592 | 0.2516 | 0.7694 |
513
+ | 0.6169 | 44500 | 0.0599 | 0.2461 | 0.7688 |
514
+ | 0.6238 | 45000 | 0.056 | 0.2438 | 0.7686 |
515
+ | 0.6307 | 45500 | 0.0573 | 0.2513 | 0.7703 |
516
+ | 0.6377 | 46000 | 0.0571 | 0.2526 | 0.7694 |
517
+ | 0.6446 | 46500 | 0.0573 | 0.2529 | 0.7702 |
518
+ | 0.6515 | 47000 | 0.0553 | 0.2529 | 0.7694 |
519
+ | 0.6585 | 47500 | 0.0541 | 0.2518 | 0.7707 |
520
+ | 0.6654 | 48000 | 0.0561 | 0.2471 | 0.7725 |
521
+ | 0.6723 | 48500 | 0.0558 | 0.2440 | 0.7710 |
522
+ | 0.6793 | 49000 | 0.0555 | 0.2556 | 0.7691 |
523
+ | 0.6862 | 49500 | 0.056 | 0.2479 | 0.7721 |
524
+ | 0.6931 | 50000 | 0.0564 | 0.2463 | 0.7723 |
525
+ | 0.7001 | 50500 | 0.0539 | 0.2561 | 0.7728 |
526
+ | 0.7070 | 51000 | 0.0526 | 0.2416 | 0.7725 |
527
+ | 0.7139 | 51500 | 0.0561 | 0.2501 | 0.7723 |
528
+ | 0.7209 | 52000 | 0.0545 | 0.2316 | 0.7732 |
529
+ | 0.7278 | 52500 | 0.0545 | 0.2352 | 0.7739 |
530
+ | 0.7347 | 53000 | 0.05 | 0.2278 | 0.7734 |
531
+ | 0.7416 | 53500 | 0.0515 | 0.2308 | 0.7730 |
532
+ | 0.7486 | 54000 | 0.0528 | 0.2524 | 0.7727 |
533
+ | 0.7555 | 54500 | 0.0509 | 0.2645 | 0.7717 |
534
+ | 0.7624 | 55000 | 0.0514 | 0.2659 | 0.7708 |
535
+ | 0.7694 | 55500 | 0.0503 | 0.2570 | 0.7725 |
536
+ | 0.7763 | 56000 | 0.0538 | 0.2524 | 0.7724 |
537
+ | 0.7832 | 56500 | 0.0477 | 0.2537 | 0.7719 |
538
+ | 0.7902 | 57000 | 0.0514 | 0.2333 | 0.7733 |
539
+ | 0.7971 | 57500 | 0.05 | 0.2420 | 0.7722 |
540
+ | 0.8040 | 58000 | 0.0518 | 0.2342 | 0.7734 |
541
+ | 0.8110 | 58500 | 0.0508 | 0.2402 | 0.7730 |
542
+ | 0.8179 | 59000 | 0.0474 | 0.2477 | 0.7711 |
543
+ | 0.8248 | 59500 | 0.0493 | 0.2465 | 0.7723 |
544
+ | 0.8318 | 60000 | 0.0492 | 0.2448 | 0.7731 |
545
+ | 0.8387 | 60500 | 0.0496 | 0.2498 | 0.7733 |
546
+ | 0.8456 | 61000 | 0.0479 | 0.2505 | 0.7721 |
547
+ | 0.8525 | 61500 | 0.0445 | 0.2449 | 0.7745 |
548
+ | **0.8595** | **62000** | **0.0477** | **0.2507** | **0.7748** |
549
+ | 0.8664 | 62500 | 0.0491 | 0.2551 | 0.7716 |
550
+ | 0.8733 | 63000 | 0.0474 | 0.2451 | 0.7743 |
551
+ | 0.8803 | 63500 | 0.0452 | 0.2464 | 0.7741 |
552
+ | 0.8872 | 64000 | 0.0482 | 0.2412 | 0.7742 |
553
+ | 0.8941 | 64500 | 0.0483 | 0.2444 | 0.7736 |
554
+ | 0.9011 | 65000 | 0.0485 | 0.2456 | 0.7724 |
555
+ | 0.9080 | 65500 | 0.045 | 0.2493 | 0.7730 |
556
+ | 0.9149 | 66000 | 0.0496 | 0.2499 | 0.7721 |
557
+ | 0.9219 | 66500 | 0.0461 | 0.2474 | 0.7748 |
558
+ | 0.9288 | 67000 | 0.0465 | 0.2432 | 0.7743 |
559
+ | 0.9357 | 67500 | 0.0477 | 0.2432 | 0.7729 |
560
+ | 0.9427 | 68000 | 0.0425 | 0.2491 | 0.7740 |
561
+ | 0.9496 | 68500 | 0.0452 | 0.2445 | 0.7736 |
562
+ | 0.9565 | 69000 | 0.046 | 0.2397 | 0.7742 |
563
+ | 0.9634 | 69500 | 0.0449 | 0.2539 | 0.7731 |
564
+ | 0.9704 | 70000 | 0.0462 | 0.2446 | 0.7745 |
565
+ | 0.9773 | 70500 | 0.0435 | 0.2385 | 0.7742 |
566
+ | 0.9842 | 71000 | 0.0469 | 0.2334 | 0.7750 |
567
+ | 0.9912 | 71500 | 0.0447 | 0.2312 | 0.7745 |
568
+ | 0.9981 | 72000 | 0.0465 | 0.2361 | 0.7737 |
569
+ | 1.0050 | 72500 | 0.0341 | 0.2359 | 0.7728 |
570
+ | 1.0120 | 73000 | 0.03 | 0.2405 | 0.7727 |
571
+ | 1.0189 | 73500 | 0.029 | 0.2241 | 0.7724 |
572
+ | 1.0258 | 74000 | 0.0284 | 0.2297 | 0.7740 |
573
+ | 1.0328 | 74500 | 0.0273 | 0.2317 | 0.7735 |
574
+ | 1.0397 | 75000 | 0.0291 | 0.2352 | 0.7727 |
575
+ | 1.0466 | 75500 | 0.0286 | 0.2439 | 0.7724 |
576
+ | 1.0536 | 76000 | 0.0268 | 0.2336 | 0.7732 |
577
+ | 1.0605 | 76500 | 0.0276 | 0.2298 | 0.7728 |
578
+ | 1.0674 | 77000 | 0.0279 | 0.2268 | 0.7726 |
579
+ | 1.0743 | 77500 | 0.0283 | 0.2206 | 0.7738 |
580
+ | 1.0813 | 78000 | 0.0277 | 0.2263 | 0.7733 |
581
+ | 1.0882 | 78500 | 0.0285 | 0.2228 | 0.7740 |
582
+ | 1.0951 | 79000 | 0.0283 | 0.2250 | 0.7729 |
583
+ | 1.1021 | 79500 | 0.0276 | 0.2200 | 0.7730 |
584
+ | 1.1090 | 80000 | 0.0276 | 0.2221 | 0.7739 |
585
+ | 1.1159 | 80500 | 0.0268 | 0.2279 | 0.7730 |
586
+ | 1.1229 | 81000 | 0.0274 | 0.2302 | 0.7733 |
587
+ | 1.1298 | 81500 | 0.0281 | 0.2286 | 0.7736 |
588
+ | 1.1367 | 82000 | 0.0267 | 0.2306 | 0.7733 |
589
+ | 1.1437 | 82500 | 0.0267 | 0.2348 | 0.7731 |
590
+ | 1.1506 | 83000 | 0.0278 | 0.2301 | 0.7729 |
591
+ | 1.1575 | 83500 | 0.028 | 0.2240 | 0.7738 |
592
+ | 1.1645 | 84000 | 0.0282 | 0.2196 | 0.7744 |
593
+ | 1.1714 | 84500 | 0.0264 | 0.2241 | 0.7737 |
594
+ | 1.1783 | 85000 | 0.0258 | 0.2252 | 0.7736 |
595
+ | 1.1852 | 85500 | 0.027 | 0.2196 | 0.7742 |
596
+ | 1.1922 | 86000 | 0.0256 | 0.2189 | 0.7739 |
597
+ | 1.1991 | 86500 | 0.0259 | 0.2174 | 0.7749 |
598
+ | 1.2060 | 87000 | 0.0262 | 0.2209 | 0.7751 |
599
+ | 1.2130 | 87500 | 0.0265 | 0.2202 | 0.7739 |
600
+ | 1.2199 | 88000 | 0.025 | 0.2228 | 0.7737 |
601
+ | 1.2268 | 88500 | 0.0266 | 0.2233 | 0.7739 |
602
+ | 1.2338 | 89000 | 0.0261 | 0.2255 | 0.7736 |
603
+ | 1.2407 | 89500 | 0.0271 | 0.2219 | 0.7746 |
604
+ | 1.2476 | 90000 | 0.0256 | 0.2185 | 0.7757 |
605
+ | 1.2546 | 90500 | 0.0257 | 0.2190 | 0.7758 |
606
+ | 1.2615 | 91000 | 0.0239 | 0.2210 | 0.7750 |
607
+ | 1.2684 | 91500 | 0.0252 | 0.2236 | 0.7743 |
608
+ | 1.2754 | 92000 | 0.0245 | 0.2238 | 0.7743 |
609
+ | 1.2823 | 92500 | 0.0267 | 0.2234 | 0.7747 |
610
+ | 1.2892 | 93000 | 0.025 | 0.2235 | 0.7746 |
611
+ | 1.2961 | 93500 | 0.0246 | 0.2298 | 0.7740 |
612
+ | 1.3031 | 94000 | 0.0266 | 0.2239 | 0.7744 |
613
+ | 1.3100 | 94500 | 0.0256 | 0.2231 | 0.7740 |
614
+ | 1.3169 | 95000 | 0.0265 | 0.2214 | 0.7744 |
615
+ | 1.3239 | 95500 | 0.0253 | 0.2221 | 0.7747 |
616
+ | 1.3308 | 96000 | 0.0251 | 0.2222 | 0.7742 |
617
+ | 1.3377 | 96500 | 0.0244 | 0.2211 | 0.7748 |
618
+ | 1.3447 | 97000 | 0.0249 | 0.2216 | 0.7750 |
619
+ | 1.3516 | 97500 | 0.0257 | 0.2215 | 0.7745 |
620
+ | 1.3585 | 98000 | 0.0263 | 0.2215 | 0.7749 |
621
+ | 1.3655 | 98500 | 0.0258 | 0.2209 | 0.7749 |
622
+ | 1.3724 | 99000 | 0.0255 | 0.2212 | 0.7748 |
623
+ | 1.3793 | 99500 | 0.0252 | 0.2213 | 0.7751 |
624
+ | 1.3863 | 100000 | 0.0257 | 0.2213 | 0.7747 |
625
+
626
+ * The bold row denotes the saved checkpoint.
627
+ </details>
628
 
629
  ### Framework Versions
630
  - Python: 3.12.3
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:448d857906393ee21e394a75e3ea8c01d7b190e2615a045ff196f66be0967d04
3
  size 298041696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1cd9a1d35025cdb46c51f16954bb7cb4e7c4e454aa300fbe6f3316645b2c4df
3
  size 298041696