dnth/ssf-retriever-modernbert-embed-base-v3

SentenceTransformer based on nomic-ai/modernbert-embed-base

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the ssf-train-valid-v3 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nomic-ai/modernbert-embed-base
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid-v3

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("dnth/ssf-retriever-modernbert-embed-base-v3")
# Run inference
sentences = [
    "The Sales Account Manager acts as a key point of contact between an organisation and its clients. He/She possesses thorough product knowledge and oversees product and/or service sales. He works with customers to identify their wants and prepares reports by collecting, analysing, and summarising sales information. He contacts existing customers to discuss and give recommendations on how specific products or services can meet their needs. He maintains customer relationships to strategically place new products and drive sales for long-term growth. He works in a fast-paced and dynamic environment, and travels frequently to clients' premises for meetings. He is familiar with client relationship management and sales tools. He is knowledgeable of the organisation's products and services, as well as trends, developments and challenges of the industry domain. The Sales Account Manager is a resourceful, people-focused and persistent individual, who takes rejection as a personal challenge to succeed when given opportunity. He appreciates the value of long lasting relationships and prioritises efforts to build trust with existing and potential customers. He exhibits good listening skills and is able to establish rapport with customers and team members alike easily.",
    'The Sales Account Manager serves as the primary liaison between the company and its clientele, leveraging deep product expertise to drive sales of products and services. This role involves collaborating closely with customers to understand their requirements, preparing detailed sales reports through data collection and analysis, and proactively reaching out to current clients to offer tailored recommendations. The Sales Account Manager nurtures client relationships with the objective of introducing new products and fostering sustainable revenue growth. Operating in a dynamic and fast-moving setting, frequent travel to client locations is expected. Proficiency in customer relationship management software and sales platforms is essential. In addition to comprehensive knowledge of the company’s offerings, the individual stays informed of industry trends and challenges. The Sales Account Manager demonstrates resilience, strong interpersonal skills, and a commitment to building trust through attentive listening and relationship cultivation.',
    'The Sales Account Manager leads a team responsible for developing marketing strategies and overseeing brand promotion within the organisation. He/She designs campaigns to increase product awareness, coordinates with advertising agencies, and manages digital content creation to enhance market presence. The role requires supervising junior marketing staff, setting performance targets, and analysing campaign metrics to adjust marketing approaches. The Sales Account Manager operates primarily from the office, with limited travel obligations, and collaborates closely with the product development team to align messaging. Strong skills in marketing analytics, campaign management tools, and creative communication are essential. This role focuses on strategic branding and promotional activities rather than direct client sales or relationship management.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9013, 0.5809],
#         [0.9013, 1.0000, 0.6224],
#         [0.5809, 0.6224, 1.0000]])

Training Details

Training Dataset

ssf-train-valid-v3

• Dataset: ssf-train-valid-v3 at f461fff
• Size: 3,016 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 57 tokens mean: 169.03 tokens max: 403 tokens	min: 55 tokens mean: 138.39 tokens max: 275 tokens	min: 37 tokens mean: 109.4 tokens max: 201 tokens

• Samples:

  anchor	positive	negative
  The Marketing Director drives the organisations business strategy by establishing the organisation's integrated marketing communications (IMC) strategy, partnership marketing arrangements and advices on product development and enhancement. He/She provides senior management with marketing advise, develops budget and manpower plans; and focuses on executing the IMC and partnership marketing plans to achieve business results. He directs the research and data analytics to obtain market and client insights, translates client insights into products and product features with market interest or potential market demand. He operates in a rapidly transforming business environment and functions through his understanding of consumers insights, market trends and industry landscape to promote the organisation and increase market demand. He is a results-oriented, astute leader who is able to negotiate strategically. He possesses strong business acumen and broad understanding of consumer, market and in...	The Marketing Director spearheads the company’s business growth by formulating and implementing a comprehensive integrated marketing communications (IMC) strategy and fostering strategic partnerships. This role advises senior leadership on marketing initiatives, oversees budget allocation and manpower planning, and ensures the successful execution of IMC and partnership campaigns to meet business objectives. The Marketing Director leads market research and data analysis efforts to capture consumer and market insights, translating these findings into innovative product developments aligned with emerging market demands. Operating within a dynamic and evolving business landscape, the director leverages deep knowledge of consumer behavior, market trends, and the competitive environment to enhance brand presence and drive demand. A decisive and visionary leader, this individual excels in strategic negotiations, demonstrates strong commercial insight, and inspires teams with a customer-centr...	The Marketing Analyst directs the organisation’s market research functions by designing and implementing data collection frameworks, analysing consumer data, and supporting the development of targeted marketing campaigns. He/She collaborates with product teams to provide data-driven recommendations for product positioning and pricing strategies. Responsible for preparing detailed reports and presentations for marketing managers, the analyst works under close supervision and focuses on executing market segmentation and consumer behavior studies. This role operates in a structured environment, requiring proficiency in data analytics tools and an understanding of marketing metrics to support decision-making. The Marketing Analyst is a detail-oriented individual with strong quantitative skills and the ability to communicate insights to internal teams but does not hold strategic leadership responsibilities or negotiate external partnerships.
  An Enrolled Nurse is responsible for providing basic nursing care and patient education under the supervision and direction of a registered nurse, in collaboration with the healthcare teams according to the established policies, procedures and guidelines. S/He attends relevant nursing training to ensure that her/his skills remain up-to-date to provide patients with quality nursing care. S/He coaches new enrolled nurses, students and support care staff. S/He operates in a wide variety of settings such as acute care, primary care, community hospitals, integrated care and long-term care facilities. S/He should be meticulous, accountable and a team player.	The Enrolled Nurse delivers fundamental nursing care and patient education while working under the guidance of a registered nurse and collaborating with multidisciplinary healthcare teams in accordance with established protocols and guidelines. This role involves participating in ongoing nursing training to maintain current competencies and ensure high-quality patient care. Additionally, the Enrolled Nurse mentors newly recruited enrolled nurses, nursing students, and support care personnel. The position operates across diverse healthcare environments, including acute hospitals, primary care centers, community hospitals, integrated care networks, and long-term care institutions. Candidates should demonstrate thoroughness, responsibility, and strong teamwork skills.	The Enrolled Nurse manages patient admission processes and medical records under the supervision of healthcare administrators, coordinating with hospital support services according to institutional policies and administrative guidelines. The role requires attending workshops on healthcare administration to keep skills current and providing training to new administrative staff and interns. This position functions primarily within hospital admissions, billing departments, outpatient clinics, and medical records offices. Attention to detail, accountability, and effective collaboration are essential for success.
  The Depot Train Controller directs the movement of trains within the depot, including launching and withdrawing trains in accordance with train service standards and requirements, and facilitating the stabling of trains in the depot for service and maintenance works. He/She coordinates with relevant internal and external stakeholders to execute first-line recovery of trains during train service disruptions. He is organised, meticulous, and systematic in managing the movement and stabling of trains, and in coordinating track access, so as to ensure the highest safety standards for personnel and train movement are upheld in the depot premises.	The Depot Train Controller oversees the scheduling and movement of trains within the depot, managing the deployment and withdrawal of trains following operational standards and service protocols. This role involves collaborating with various internal teams and external partners to facilitate initial recovery actions during service interruptions. The controller must be detail-oriented, methodical, and well-organized when handling train positioning and track allocation to maintain strict safety compliance for both staff and train operations within the depot.	The Depot Maintenance Planner coordinates the scheduling of routine and emergency maintenance activities for trains across the network, ensuring maintenance tasks align with safety regulations and service availability. This role requires liaising with engineering teams and suppliers to optimize resource allocation and minimize downtime. The planner must be proactive, analytical, and adept at balancing maintenance priorities while adhering to compliance standards for depot facilities.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

ssf-train-valid-v3

• Dataset: ssf-train-valid-v3 at f461fff
• Size: 754 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 754 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 57 tokens mean: 169.99 tokens max: 352 tokens	min: 59 tokens mean: 138.43 tokens max: 285 tokens	min: 34 tokens mean: 109.81 tokens max: 274 tokens

• Samples:

  anchor	positive	negative
  The Quality Control Laboratory Analyst/Chemist/Microbiologist monitors sampling, cleanliness and product quality testing activities, performs non-standard quality tests, and manages associated documentation and data. He/She identifies the operating criteria for the tools, equipment and materials to be used, and collaborates with the Engineering and Maintenance department to ensure that laboratory equipment and infrastructure function as required. In addition, he/ implements Standard Operating Procedures (SOPs) and workflow improvements in the laboratory. The Quality Control Laboratory Analyst/Chemist/Microbiologist works in a laboratory setting, primarily in a cleanroom environment, and may be required to work on a shift. He has to exercise critical and analytical thinking to review data and identify discrepancies against set criteria. He requires strong communication and teamwork to collaborate effectively with others in order to fulfil work objectives.	The Quality Control Laboratory Analyst/Chemist/Microbiologist is responsible for overseeing sampling processes, ensuring cleanliness standards, and conducting product quality assessments, including specialized testing procedures. This role involves determining optimal operating parameters for laboratory instruments, coordinating with Engineering and Maintenance teams to maintain equipment functionality, and enforcing Standard Operating Procedures (SOPs) alongside continuous workflow enhancements. Operating mainly within a cleanroom laboratory environment, often on rotational shifts, the analyst must apply strong analytical skills to evaluate data accuracy and detect inconsistencies. Effective communication and collaborative teamwork are essential to achieving quality assurance objectives.	The Quality Control Laboratory Technician coordinates sampling and cleanliness checks, but primarily focuses on maintaining inventory and ordering laboratory supplies. This position works closely with procurement and logistics teams rather than Engineering or Maintenance, ensuring that laboratory consumables are stocked and equipment calibration schedules are tracked. The technician operates in a standard laboratory space and follows established SOPs without involvement in workflow improvements or critical data analysis. While teamwork is necessary for administrative tasks, the role requires minimal analytical judgment and no shift work is typically expected.
  The Assistant Keeper/Assistant Aquarist assists in the care and management of wildlife within the organisation/attractions sites. This includes supporting the preparation of food to the wildlife, caring for ill animals, checking enclosures and cages for signs of wear or damage for animal, staff and visitor safety, and giving educational talks/tours to the visitors. He/she also assists in maintaining animal training behaviours and promoting conservation awareness through animal presentations. Conscientious and responsible, he is attentive to the needs of the wildlife under his care, and leverages his strong observation skills to monitor and report the status of characteristics and behaviours of the wildlife under his care. He is able to work both independently and under direction. He is physically fit and works in a shift system encompassing weekends and public holidays. Outside the working hours, he may be on a rota for call-outs. He often stays outdoors for long periods of time even t...	The Assistant Keeper/Assistant Aquarist supports the daily care and management of animals at wildlife facilities or attraction sites. Responsibilities include assisting with feeding routines, providing care for sick animals, inspecting enclosures for safety hazards, and delivering educational tours to visitors. This role involves helping to maintain animal training programs and raising public awareness about conservation through animal demonstrations. The assistant is diligent and attentive, closely observing animal health and behavior to report any changes. Capable of working independently or under supervision, the incumbent must be physically resilient, work shifts including weekends and public holidays, and be prepared for on-call duties. The position often requires extended outdoor work in various weather conditions and may necessitate a valid driving licence or scuba-diving certification depending on the workplace environment.	The Animal Nutritionist develops dietary plans and nutritional programs for wildlife and captive animals within zoological and conservation organizations. This role focuses on formulating balanced diets, analyzing feed components, and collaborating with veterinary teams to optimize animal health through specialized nutrition. The Animal Nutritionist primarily works in laboratory and office settings, conducting research and evaluating the effects of various feed formulations. Unlike hands-on caretaking roles, this position requires advanced knowledge of animal physiology and dietetics, with minimal direct interaction with visitors or enclosure maintenance. This role typically operates during standard office hours and does not involve shift work or outdoor duties.
  The Senior Manager works in the field of counselling management. He/She should be qualified and trained to monitor and manage the organisation's strategic initiative, resource management, collaboration and corporate governance to ensure operational efficiency. He oversees strategic implementation, budgets the use of capital and human resources, develops professional development programmes, and initiates professional relationships across agencies. He also ensures operational and governance efficiency through supervision of a multi-disciplinary staff performance evaluation, and policy implementation. He is an experienced management staff who is meticulous, committed and possesses good problem-solving skills.	The Senior Manager in counselling leadership is responsible for guiding and overseeing the organisation’s strategic priorities, resource allocation, and inter-agency partnerships to maximise operational effectiveness. This role entails supervising the execution of strategic plans, managing budgets for both financial and human capital, designing continuing professional development initiatives, and fostering collaborative networks across different agencies. The Senior Manager also ensures adherence to governance policies and evaluates the performance of a multidisciplinary team. This position requires an experienced, detail-oriented leader with strong commitment and advanced problem-solving capabilities.	The Senior Manager in community health administration leads initiatives in healthcare program management and service delivery. He/She coordinates healthcare resources, manages patient care budgets, and develops training programs for medical staff. The role involves overseeing clinical operations, ensuring compliance with health regulations, and supervising multidisciplinary healthcare teams. This professional must be highly experienced, organized, and skilled in resolving complex clinical and administrative issues.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: False
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	6	0.1655	0.0146
2.0	12	0.0099	0.0039
3.0	18	0.0056	0.0027
4.0	24	0.0046	0.0023
5.0	30	0.0042	0.0022

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.8
• Sentence Transformers: 5.1.0
• Transformers: 4.55.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}