luiggy2620/distiluse-es-legal-balanced-final

distiluse-es-legal-embeddings

This is a sentence-transformers model finetuned from sentence-transformers/distiluse-base-multilingual-cased-v2. It maps sentences & paragraphs to a 512-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/distiluse-base-multilingual-cased-v2
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 512 dimensions
• Similarity Function: Cosine Similarity
• Language: es
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False, 'architecture': 'DistilBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 512, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("luiggy2620/distiluse-es-legal-balanced-final")
# Run inference
sentences = [
    'TERCERA. OBJETO. LA VENDEDORA de su libre y espontánea voluntad, sin que medie presión alguna, dolo u otro vicio de consentimiento y por convenir así a sus intereses, da en venta real y definitiva la totalidad del inmueble a LA COMPRADORA, por el precio bilateralmente convenido de precio_1 que LA VENDEDORA, declara recibir en su integridad, en efectivo, en moneda de curso legal y a su entera satisfacción a tiempo de suscribir el presente contrato.',
    '¿Qué vende la vendedora a la compradora?',
    'Qué tipos de sellos aparecen al margen de la minuta?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 512]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.3758, -0.0276],
#         [ 0.3758,  1.0000, -0.0335],
#         [-0.0276, -0.0335,  1.0000]])

Evaluation

Metrics

Triplet

• Dataset: triplet_eval
• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	1.0

Training Details

Training Dataset

Unnamed Dataset

• Size: 5,270 training samples
• Columns: sentence0 and sentence1
• Approximate statistics based on the first 1000 samples:

  	sentence0	sentence1
  type	string	string
  details	min: 13 tokens mean: 100.1 tokens max: 128 tokens	min: 9 tokens mean: 26.94 tokens max: 128 tokens

• Samples:

  sentence0	sentence1
  numero_de_documento_1 Esc. Pub de Traspaso De Vehículo placa: placa_de_vehiculo_1 propiedad de persona_1 para la venta a persona_3 por precio_1	El precio de venta del vehículo es precio_1
  numero_de_documento_2 ESCRITURA PÚBLICA DE transacción de vehículo motorizado, PLACA Nº placa_de_vehiculo_3 OTORGADA POR: persona_4 en representación de: persona_2 A FAVOR DE: persona_4 PROPIETARIO DE LA empresa_unipersonal_1 POR EL MONTO DE precio_1 CONVENIDO.	El numero y año del documento es numero_de_documento_1
  numero_de_documento_4 ESCRITURA PÚBLICA DE Adquisición De Un Vehículo, PLACA Nº placa_de_vehiculo_1 OTORGADA POR: persona_1 PROPIETARIO DE LA empresa_unipersonal_3 A FAVOR DE: persona_2 persona_3 Y persona_4 SOCIOS Y REPRESENTANTES DE LA sociedad_srl_4 POR LA SUMA DE precio_2	La escritura pública es sobre una venta de vehiculo de un vehículo.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 930 evaluation samples
• Columns: sentence0 and sentence1
• Approximate statistics based on the first 930 samples:

  	sentence0	sentence1
  type	string	string
  details	min: 19 tokens mean: 97.72 tokens max: 128 tokens	min: 7 tokens mean: 25.75 tokens max: 128 tokens

• Samples:

  sentence0	sentence1
  numero_de_documento_5 ESCRITURA PÚBLICA DE TRANSACCIÓN DE VEHÍCULO MOTORIZADO, PLACA NRO. placa_de_vehiculo_1 SUSCRITA POR LOS SEÑORES: persona_2 persona_2 Y persona_1 EN BENEFICIO DE: persona_4 POR LA CANTIDAD DE precio_5	¿Quiénes suscribieron la transacción del vehículo motorizado?
  numero_de_documento_1 ESCRITURA PÚBLICA DE adjudicacion de vehículo motorizado CONTRATANDO CONSIGO MISMO, PLACA DE CIRCULACIÓN Nº placa_de_vehiculo_3 QUE SUSCRIBE: persona_1 por si y en representación de: persona_4 A FAVOR DE SI MISMO EN CONFORMIDAD AL ART. codigo_1 DEL CODIGO CIVIL, POR EL PRECIO DE precio_1	¿Cuál es el precio del vehículo?
  numero_de_documento_1 escritura pública de: Compraventa De Un Automóvil, placa_de_vehiculo_4 OTORGADO POR: persona_1 A FAVOR DE: persona_2 POR EL PRECIO DE precio_1	El comprador y nuevo propietario del vehiculo es persona_1

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 20
• learning_rate: 8e-06
• weight_decay: 0.03
• max_grad_norm: 0.3
• lr_scheduler_type: cosine
• warmup_ratio: 0.2
• fp16: True
• dataloader_num_workers: 1
• load_best_model_at_end: True
• dataloader_pin_memory: False

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 20
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 8e-06
• weight_decay: 0.03
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 0.3
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.2
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 1
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: False
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	triplet_eval_cosine_accuracy
-1	-1	-	-	0.5400
0.0095	5	2.4306	-	-
0.0190	10	2.3122	-	-
0.0285	15	2.2741	-	-
0.0380	20	2.0905	-	-
0.0474	25	2.2929	-	-
0.0095	5	2.1049	-	-
0.0190	10	2.0305	-	-
0.0285	15	1.7969	-	-
0.0380	20	1.7819	-	-
0.0474	25	2.046	2.3840	0.6750
0.0569	30	1.3209	-	-
0.0664	35	1.3396	-	-
0.0759	40	1.5147	-	-
0.0854	45	1.4947	-	-
0.0949	50	1.4001	1.9371	0.7750
0.1044	55	1.5686	-	-
0.1139	60	0.828	-	-
0.1233	65	1.5542	-	-
0.1328	70	1.4292	-	-
0.1423	75	0.8558	1.6839	0.8100
0.1518	80	1.1573	-	-
0.1613	85	0.9952	-	-
0.1708	90	1.1084	-	-
0.1803	95	1.2861	-	-
0.1898	100	1.3884	1.5355	0.8400
0.1992	105	1.1502	-	-
0.2087	110	1.2771	-	-
0.2182	115	1.1399	-	-
0.2277	120	1.0	-	-
0.2372	125	1.0145	1.4050	0.8500
0.2467	130	0.705	-	-
0.2562	135	0.684	-	-
0.2657	140	1.0563	-	-
0.2751	145	1.1791	-	-
0.2846	150	0.6746	1.3098	0.9000
0.2941	155	1.0281	-	-
0.3036	160	0.8901	-	-
0.3131	165	0.5872	-	-
0.3226	170	0.6582	-	-
0.3321	175	0.788	1.2584	0.8950
0.3416	180	0.7405	-	-
0.3510	185	0.7737	-	-
0.3605	190	0.8017	-	-
0.3700	195	0.849	-	-
0.3795	200	0.9972	1.1327	0.9400
0.3890	205	0.8374	-	-
0.3985	210	0.6933	-	-
0.4080	215	0.5667	-	-
0.4175	220	0.5733	-	-
0.4269	225	0.4814	1.1721	0.9450
0.4364	230	0.7727	-	-
0.4459	235	0.7623	-	-
0.4554	240	0.7466	-	-
0.4649	245	0.7489	-	-
0.4744	250	0.7185	1.0541	0.9500
0.4839	255	0.6422	-	-
0.4934	260	0.4478	-	-
0.5028	265	0.8566	-	-
0.5123	270	0.5825	-	-
0.5218	275	0.6335	1.0094	0.9550
0.5313	280	0.4719	-	-
0.5408	285	0.7613	-	-
0.5503	290	0.6041	-	-
0.5598	295	0.7514	-	-
0.5693	300	0.693	0.9468	0.9550
0.5787	305	0.6485	-	-
0.5882	310	0.4796	-	-
0.5977	315	0.3809	-	-
0.6072	320	0.602	-	-
0.6167	325	0.4706	0.9221	0.9650
0.6262	330	0.4818	-	-
0.6357	335	0.5331	-	-
0.6452	340	0.5715	-	-
0.6546	345	0.5022	-	-
0.6641	350	0.8457	0.9027	0.9500
0.6736	355	0.7912	-	-
0.6831	360	0.657	-	-
0.6926	365	0.496	-	-
0.7021	370	0.5524	-	-
0.7116	375	0.4065	0.8646	0.9750
0.7211	380	0.3708	-	-
0.7306	385	0.6628	-	-
0.7400	390	0.5983	-	-
0.7495	395	0.445	-	-
0.7590	400	0.6094	0.8604	0.9750
0.7685	405	0.3963	-	-
0.7780	410	0.6161	-	-
0.7875	415	0.3776	-	-
0.7970	420	0.6449	-	-
0.8065	425	0.5054	0.7672	0.9800
0.8159	430	0.5518	-	-
0.8254	435	0.3351	-	-
0.8349	440	0.6461	-	-
0.8444	445	0.3986	-	-
0.8539	450	0.5226	0.7484	0.9850
0.8634	455	0.7118	-	-
0.8729	460	0.4228	-	-
0.8824	465	0.6646	-	-
0.8918	470	0.3974	-	-
0.9013	475	0.4638	0.7319	0.9700
0.9108	480	0.3994	-	-
0.9203	485	0.351	-	-
0.9298	490	0.2993	-	-
0.9393	495	0.6136	-	-
0.9488	500	0.5763	0.7545	0.9700
0.9583	505	0.4826	-	-
0.9677	510	0.4333	-	-
0.9772	515	0.6736	-	-
0.9867	520	0.5241	-	-
0.9962	525	0.3101	0.6897	0.9650
1.0057	530	0.474	-	-
1.0152	535	0.2346	-	-
1.0247	540	0.3659	-	-
1.0342	545	0.3046	-	-
1.0436	550	0.4148	0.7027	0.9800
1.0531	555	0.321	-	-
1.0626	560	0.4315	-	-
1.0721	565	0.4838	-	-
1.0816	570	0.2886	-	-
1.0911	575	0.3718	0.6581	0.9800
1.1006	580	0.2237	-	-
1.1101	585	0.3343	-	-
1.1195	590	0.3573	-	-
1.1290	595	0.2421	-	-
1.1385	600	0.3315	0.6553	0.9750
1.1480	605	0.3258	-	-
1.1575	610	0.714	-	-
1.1670	615	0.244	-	-
1.1765	620	0.3899	-	-
1.1860	625	0.2789	0.6135	0.9850
1.1954	630	0.2721	-	-
1.2049	635	0.0957	-	-
1.2144	640	0.3631	-	-
1.2239	645	0.6304	-	-
1.2334	650	0.1841	0.6325	0.9800
1.2429	655	0.2118	-	-
1.2524	660	0.3037	-	-
1.2619	665	0.3958	-	-
1.2713	670	0.5982	-	-
1.2808	675	0.5016	0.6113	0.9850
1.2903	680	0.5266	-	-
1.2998	685	0.4622	-	-
1.3093	690	0.3678	-	-
1.3188	695	0.3591	-	-
1.3283	700	0.3461	0.5873	0.9800
1.3378	705	0.2499	-	-
1.3472	710	0.3621	-	-
1.3567	715	0.4742	-	-
1.3662	720	0.2846	-	-
1.3757	725	0.4233	0.5780	0.9850
1.3852	730	0.5183	-	-
1.3947	735	0.4594	-	-
1.4042	740	0.3213	-	-
1.4137	745	0.479	-	-
1.4231	750	0.244	0.5756	0.9800
1.4326	755	0.3029	-	-
1.4421	760	0.3128	-	-
1.4516	765	0.2308	-	-
1.4611	770	0.2453	-	-
1.4706	775	0.4089	0.5459	0.9850
1.4801	780	0.399	-	-
1.4896	785	0.3846	-	-
1.4991	790	0.2698	-	-
1.5085	795	0.7337	-	-
1.5180	800	0.4169	0.5220	0.9850
1.5275	805	0.2407	-	-
1.5370	810	0.0888	-	-
1.5465	815	0.4584	-	-
1.5560	820	0.2206	-	-
1.5655	825	0.2638	0.5339	0.9850
1.5750	830	0.3155	-	-
1.5844	835	0.2733	-	-
1.5939	840	0.3092	-	-
1.6034	845	0.3355	-	-
1.6129	850	0.1425	0.5274	0.9900
1.6224	855	0.2576	-	-
1.6319	860	0.3539	-	-
1.6414	865	0.3728	-	-
1.6509	870	0.3114	-	-
1.6603	875	0.3883	0.5148	0.9900
1.6698	880	0.1344	-	-
1.6793	885	0.2097	-	-
1.6888	890	0.1263	-	-
1.6983	895	0.2317	-	-
1.7078	900	0.2354	0.5078	0.9900
1.7173	905	0.2546	-	-
1.7268	910	0.1656	-	-
1.7362	915	0.5719	-	-
1.7457	920	0.3647	-	-
1.7552	925	0.2817	0.5032	0.9900
1.7647	930	0.2482	-	-
1.7742	935	0.3697	-	-
1.7837	940	0.2082	-	-
1.7932	945	0.2497	-	-
1.8027	950	0.1688	0.5051	0.9900
1.8121	955	0.4251	-	-
1.8216	960	0.3603	-	-
1.8311	965	0.4506	-	-
1.8406	970	0.2303	-	-
1.8501	975	0.4086	0.5111	0.9900
1.8596	980	0.3422	-	-
1.8691	985	0.0954	-	-
1.8786	990	0.3552	-	-
1.8880	995	0.4798	-	-
1.8975	1000	0.2623	0.4713	0.9950
1.9070	1005	0.2642	-	-
1.9165	1010	0.1423	-	-
1.9260	1015	0.3493	-	-
1.9355	1020	0.2362	-	-
1.9450	1025	0.1921	0.4608	0.9950
1.9545	1030	0.3555	-	-
1.9639	1035	0.1268	-	-
1.9734	1040	0.3715	-	-
1.9829	1045	0.5015	-	-
1.9924	1050	0.2288	0.4471	0.9950
2.0019	1055	0.2191	-	-
2.0114	1060	0.2769	-	-
2.0209	1065	0.2275	-	-
2.0304	1070	0.1573	-	-
2.0398	1075	0.1818	0.4448	0.9950
2.0493	1080	0.2495	-	-
2.0588	1085	0.1605	-	-
2.0683	1090	0.406	-	-
2.0778	1095	0.0975	-	-
2.0873	1100	0.1841	0.4441	0.9950
2.0968	1105	0.1031	-	-
2.1063	1110	0.1256	-	-
2.1157	1115	0.2786	-	-
2.1252	1120	0.1269	-	-
2.1347	1125	0.089	0.4454	0.9950
2.1442	1130	0.3174	-	-
2.1537	1135	0.1623	-	-
2.1632	1140	0.2372	-	-
2.1727	1145	0.2399	-	-
2.1822	1150	0.304	0.4508	0.9950
2.1917	1155	0.1913	-	-
2.2011	1160	0.1075	-	-
2.2106	1165	0.2845	-	-
2.2201	1170	0.1191	-	-
2.2296	1175	0.1083	0.4366	1.0
2.2391	1180	0.4736	-	-
2.2486	1185	0.2507	-	-
2.2581	1190	0.0838	-	-
2.2676	1195	0.1461	-	-
2.2770	1200	0.0672	0.4255	1.0
2.2865	1205	0.3276	-	-
2.2960	1210	0.2339	-	-
2.3055	1215	0.3573	-	-
2.3150	1220	0.2552	-	-
2.3245	1225	0.0565	0.4214	1.0
2.3340	1230	0.131	-	-
2.3435	1235	0.2276	-	-
2.3529	1240	0.2008	-	-
2.3624	1245	0.085	-	-
2.3719	1250	0.1554	0.4203	1.0
2.3814	1255	0.2493	-	-
2.3909	1260	0.2439	-	-
2.4004	1265	0.1253	-	-
2.4099	1270	0.2398	-	-
2.4194	1275	0.3121	0.4199	1.0
2.4288	1280	0.0742	-	-
2.4383	1285	0.1452	-	-
2.4478	1290	0.1388	-	-
2.4573	1295	0.2693	-	-
2.4668	1300	0.1332	0.4172	1.0
2.4763	1305	0.1463	-	-
2.4858	1310	0.2751	-	-
2.4953	1315	0.2274	-	-
2.5047	1320	0.1767	-	-
2.5142	1325	0.2019	0.4164	1.0
2.5237	1330	0.2877	-	-
2.5332	1335	0.2388	-	-
2.5427	1340	0.2267	-	-
2.5522	1345	0.1964	-	-
2.5617	1350	0.1779	0.4124	1.0
2.5712	1355	0.1913	-	-
2.5806	1360	0.2433	-	-
2.5901	1365	0.2963	-	-
2.5996	1370	0.1221	-	-
2.6091	1375	0.2104	0.4110	1.0
2.6186	1380	0.1383	-	-
2.6281	1385	0.1095	-	-
2.6376	1390	0.1891	-	-
2.6471	1395	0.171	-	-
2.6565	1400	0.1868	0.4102	1.0
2.6660	1405	0.0713	-	-
2.6755	1410	0.1479	-	-
2.6850	1415	0.1991	-	-
2.6945	1420	0.1648	-	-
2.7040	1425	0.2252	0.4097	1.0
2.7135	1430	0.3373	-	-
2.7230	1435	0.2842	-	-
2.7324	1440	0.2238	-	-
2.7419	1445	0.3285	-	-
2.7514	1450	0.1781	0.4097	1.0
2.7609	1455	0.2597	-	-
2.7704	1460	0.3764	-	-
2.7799	1465	0.1916	-	-
2.7894	1470	0.225	-	-
2.7989	1475	0.1697	0.4094	1.0
2.8083	1480	0.1214	-	-
2.8178	1485	0.2059	-	-
2.8273	1490	0.2136	-	-
2.8368	1495	0.2679	-	-
2.8463	1500	0.172	0.4092	1.0
2.8558	1505	0.1697	-	-
2.8653	1510	0.26	-	-
2.8748	1515	0.3535	-	-
2.8843	1520	0.1989	-	-
2.8937	1525	0.1571	0.4089	1.0
2.9032	1530	0.2165	-	-
2.9127	1535	0.2085	-	-
2.9222	1540	0.2513	-	-
2.9317	1545	0.2036	-	-
2.9412	1550	0.0774	0.4088	1.0
2.9507	1555	0.1856	-	-
2.9602	1560	0.3345	-	-
2.9696	1565	0.2177	-	-
2.9791	1570	0.1433	-	-
2.9886	1575	0.2396	0.4088	1.0
2.9981	1580	0.1049	-	-
-1	-1	-	-	1.0

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 5.0.0
• Transformers: 4.54.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.9.0
• Datasets: 4.0.0
• Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}