dnth/ssf-reranker-modernbert-embed-base-v0

CrossEncoder based on answerdotai/ModernBERT-base

This is a Cross Encoder model finetuned from answerdotai/ModernBERT-base using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: answerdotai/ModernBERT-base
• Maximum Sequence Length: 8192 tokens
• Number of Output Labels: 1 label

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("dnth/ssf-reranker-modernbert-embed-base-v0")
# Get scores for pairs of texts
pairs = [
    ['The Audit Associate/Audit Assistant Associate undertakes specific stages of audit work under supervision. He/She begins to appreciate the underlying principles behind the tasks assigned to him as part of the audit plan. He is also able to make adjustments to the application of skills to improve the work tasks or solve non-complex issues. The Audit Associate/Audit Assistant Associate operates in a structured work environment. He is able to build relationships, work in a team and identify ethical issues with reference to the code of professional conduct and ethics. He is able to select and apply from a range of known solutions to familiar problems and takes responsibility for his own learning and performance. He is a trustworthy and meticulous individual.', 'The Audit Assistant is responsible for carrying out designated audit procedures under guidance. They develop an understanding of the core concepts behind their assigned tasks within the audit framework. The Audit Assistant can adapt their skills to enhance task execution or address straightforward challenges. Working in a well-organized setting, they collaborate effectively with colleagues, recognize ethical considerations according to professional standards, and choose appropriate solutions for routine issues. They are committed to continuous learning and maintaining high standards of accuracy and integrity.'],
    ['The Audit Associate/Audit Assistant Associate undertakes specific stages of audit work under supervision. He/She begins to appreciate the underlying principles behind the tasks assigned to him as part of the audit plan. He is also able to make adjustments to the application of skills to improve the work tasks or solve non-complex issues. The Audit Associate/Audit Assistant Associate operates in a structured work environment. He is able to build relationships, work in a team and identify ethical issues with reference to the code of professional conduct and ethics. He is able to select and apply from a range of known solutions to familiar problems and takes responsibility for his own learning and performance. He is a trustworthy and meticulous individual.', 'The Marketing Coordinator plans and executes promotional campaigns to increase brand awareness. They collaborate with creative teams to design advertisements and manage social media content. The role involves analyzing market trends, coordinating events, and engaging with customers to enhance product visibility. The Marketing Coordinator thrives in a dynamic environment, demonstrates strong communication skills, and focuses on driving sales growth through innovative strategies.'],
    ['The Audit Senior Manager/Audit Manager manages a portfolio of engagements to deliver high quality audit services. He/she also provides leadership on audit engagements which includes client acceptance process, engagement planning, execution and finalisation of an audit engagement. He is fully accountable for the audit engagement and ensures that the engagement progress against budget and timeline is closely monitored. He also serves to develop and maintain long-term client relationships and value-add to the audit firm by identifying new business development opportunities. The Audit Senior Manager/Audit Manager reviews and provides key technical expertise to ensure the quality of audit work performed is in compliance with professional standards and requirements. He contributes towards continuous improvement in audit methodology and process. He will also assume a greater role in professional development activities such as training, staff recruitment and resource planning.', 'The Audit Senior Manager/Audit Manager oversees multiple audit engagements to ensure the delivery of high-quality audit services. He/she leads the entire audit process, including client acceptance, engagement planning, execution, and completion. The manager is fully responsible for the engagement, monitoring progress against budget and timelines closely. He/she also fosters long-term client relationships and identifies opportunities for business growth. Additionally, the Audit Senior Manager/Audit Manager reviews audit work for compliance with professional standards and contributes to enhancing audit methodologies and processes. The role also includes involvement in staff development, recruitment, and resource allocation.'],
    ['The Audit Senior Manager/Audit Manager manages a portfolio of engagements to deliver high quality audit services. He/she also provides leadership on audit engagements which includes client acceptance process, engagement planning, execution and finalisation of an audit engagement. He is fully accountable for the audit engagement and ensures that the engagement progress against budget and timeline is closely monitored. He also serves to develop and maintain long-term client relationships and value-add to the audit firm by identifying new business development opportunities. The Audit Senior Manager/Audit Manager reviews and provides key technical expertise to ensure the quality of audit work performed is in compliance with professional standards and requirements. He contributes towards continuous improvement in audit methodology and process. He will also assume a greater role in professional development activities such as training, staff recruitment and resource planning.', 'The Marketing Specialist develops and implements advertising campaigns to promote products and services. He/she conducts market research to identify customer needs and preferences, coordinates with creative teams to produce promotional materials, and manages social media channels to increase brand awareness. The specialist also analyzes campaign performance data to optimize marketing strategies and collaborates with sales teams to boost product sales. Additionally, the role involves organizing events and managing budgets for marketing activities.'],
    ['The Audit Partner/Audit Director is a transformational leader who steers the organisation to achieve its business goals and objectives by formulating technical and strategic directions to drive change. He/She provides strategic vision and leadership to the organisation in order to develop and strengthen organisational capabilities and culture. The Audit Partner/Audit Director is expected to promote new ideas and business solutions that result in extended services to existing clients. He constantly seeks to expand client base and support business development activities. He also establishes consistent and rigorous quality and risk management processes and procedures. The Audit Partner/Audit Director uses a multitude of controls and procedures consisting professional, regulatory, business, economic, social and environmental conditions to manage risk exposure.', 'The Audit Partner/Audit Director plays a key leadership role in guiding the organisation towards achieving its strategic objectives by setting clear technical and business directions. This leader fosters a culture of innovation and excellence, driving continuous improvement in organisational capabilities. They actively promote new business initiatives and service offerings to existing clients while expanding the client portfolio through effective business development efforts. Additionally, the Audit Partner/Audit Director implements robust quality assurance and risk management frameworks to ensure compliance with professional, regulatory, and environmental standards, thereby mitigating potential risks.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'The Audit Associate/Audit Assistant Associate undertakes specific stages of audit work under supervision. He/She begins to appreciate the underlying principles behind the tasks assigned to him as part of the audit plan. He is also able to make adjustments to the application of skills to improve the work tasks or solve non-complex issues. The Audit Associate/Audit Assistant Associate operates in a structured work environment. He is able to build relationships, work in a team and identify ethical issues with reference to the code of professional conduct and ethics. He is able to select and apply from a range of known solutions to familiar problems and takes responsibility for his own learning and performance. He is a trustworthy and meticulous individual.',
    [
        'The Audit Assistant is responsible for carrying out designated audit procedures under guidance. They develop an understanding of the core concepts behind their assigned tasks within the audit framework. The Audit Assistant can adapt their skills to enhance task execution or address straightforward challenges. Working in a well-organized setting, they collaborate effectively with colleagues, recognize ethical considerations according to professional standards, and choose appropriate solutions for routine issues. They are committed to continuous learning and maintaining high standards of accuracy and integrity.',
        'The Marketing Coordinator plans and executes promotional campaigns to increase brand awareness. They collaborate with creative teams to design advertisements and manage social media content. The role involves analyzing market trends, coordinating events, and engaging with customers to enhance product visibility. The Marketing Coordinator thrives in a dynamic environment, demonstrates strong communication skills, and focuses on driving sales growth through innovative strategies.',
        'The Audit Senior Manager/Audit Manager oversees multiple audit engagements to ensure the delivery of high-quality audit services. He/she leads the entire audit process, including client acceptance, engagement planning, execution, and completion. The manager is fully responsible for the engagement, monitoring progress against budget and timelines closely. He/she also fosters long-term client relationships and identifies opportunities for business growth. Additionally, the Audit Senior Manager/Audit Manager reviews audit work for compliance with professional standards and contributes to enhancing audit methodologies and processes. The role also includes involvement in staff development, recruitment, and resource allocation.',
        'The Marketing Specialist develops and implements advertising campaigns to promote products and services. He/she conducts market research to identify customer needs and preferences, coordinates with creative teams to produce promotional materials, and manages social media channels to increase brand awareness. The specialist also analyzes campaign performance data to optimize marketing strategies and collaborates with sales teams to boost product sales. Additionally, the role involves organizing events and managing budgets for marketing activities.',
        'The Audit Partner/Audit Director plays a key leadership role in guiding the organisation towards achieving its strategic objectives by setting clear technical and business directions. This leader fosters a culture of innovation and excellence, driving continuous improvement in organisational capabilities. They actively promote new business initiatives and service offerings to existing clients while expanding the client portfolio through effective business development efforts. Additionally, the Audit Partner/Audit Director implements robust quality assurance and risk management frameworks to ensure compliance with professional, regulatory, and environmental standards, thereby mitigating potential risks.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Training Details

Training Dataset

Unnamed Dataset

• Size: 11,310 training samples
• Columns: text1, text2, and label
• Approximate statistics based on the first 1000 samples:

  	text1	text2	label
  type	string	string	float
  details	min: 334 characters mean: 1059.22 characters max: 2396 characters	min: 183 characters mean: 789.05 characters max: 1943 characters	min: 0.0 mean: 0.5 max: 1.0

• Samples:

  text1	text2	label
  The Audit Associate/Audit Assistant Associate undertakes specific stages of audit work under supervision. He/She begins to appreciate the underlying principles behind the tasks assigned to him as part of the audit plan. He is also able to make adjustments to the application of skills to improve the work tasks or solve non-complex issues. The Audit Associate/Audit Assistant Associate operates in a structured work environment. He is able to build relationships, work in a team and identify ethical issues with reference to the code of professional conduct and ethics. He is able to select and apply from a range of known solutions to familiar problems and takes responsibility for his own learning and performance. He is a trustworthy and meticulous individual.	The Audit Assistant is responsible for carrying out designated audit procedures under guidance. They develop an understanding of the core concepts behind their assigned tasks within the audit framework. The Audit Assistant can adapt their skills to enhance task execution or address straightforward challenges. Working in a well-organized setting, they collaborate effectively with colleagues, recognize ethical considerations according to professional standards, and choose appropriate solutions for routine issues. They are committed to continuous learning and maintaining high standards of accuracy and integrity.	1.0
  The Audit Associate/Audit Assistant Associate undertakes specific stages of audit work under supervision. He/She begins to appreciate the underlying principles behind the tasks assigned to him as part of the audit plan. He is also able to make adjustments to the application of skills to improve the work tasks or solve non-complex issues. The Audit Associate/Audit Assistant Associate operates in a structured work environment. He is able to build relationships, work in a team and identify ethical issues with reference to the code of professional conduct and ethics. He is able to select and apply from a range of known solutions to familiar problems and takes responsibility for his own learning and performance. He is a trustworthy and meticulous individual.	The Marketing Coordinator plans and executes promotional campaigns to increase brand awareness. They collaborate with creative teams to design advertisements and manage social media content. The role involves analyzing market trends, coordinating events, and engaging with customers to enhance product visibility. The Marketing Coordinator thrives in a dynamic environment, demonstrates strong communication skills, and focuses on driving sales growth through innovative strategies.	0.0
  The Audit Senior Manager/Audit Manager manages a portfolio of engagements to deliver high quality audit services. He/she also provides leadership on audit engagements which includes client acceptance process, engagement planning, execution and finalisation of an audit engagement. He is fully accountable for the audit engagement and ensures that the engagement progress against budget and timeline is closely monitored. He also serves to develop and maintain long-term client relationships and value-add to the audit firm by identifying new business development opportunities. The Audit Senior Manager/Audit Manager reviews and provides key technical expertise to ensure the quality of audit work performed is in compliance with professional standards and requirements. He contributes towards continuous improvement in audit methodology and process. He will also assume a greater role in professional development activities such as training, staff recruitment and resource planning.	The Audit Senior Manager/Audit Manager oversees multiple audit engagements to ensure the delivery of high-quality audit services. He/she leads the entire audit process, including client acceptance, engagement planning, execution, and completion. The manager is fully responsible for the engagement, monitoring progress against budget and timelines closely. He/she also fosters long-term client relationships and identifies opportunities for business growth. Additionally, the Audit Senior Manager/Audit Manager reviews audit work for compliance with professional standards and contributes to enhancing audit methodologies and processes. The role also includes involvement in staff development, recruitment, and resource allocation.	1.0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• gradient_accumulation_steps: 4
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• bf16: True
• gradient_checkpointing: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Framework Versions

• Python: 3.12.8
• Sentence Transformers: 5.1.0
• Transformers: 4.55.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}