Hugging Face's logo

yosriku
/
Indobert-Base-p2-Trash-Large-EXP2

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
dense
Generated from Trainer
dataset_size:7314
loss:MultipleNegativesRankingLoss
text-embeddings-inference
Model card Files
xet
Community

SentenceTransformer based on indobenchmark/indobert-base-p2

This is a sentence-transformers model finetuned from indobenchmark/indobert-base-p2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: indobenchmark/indobert-base-p2
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 768 dimensions
  • Similarity Function: Cosine Similarity

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("yosriku/Indobert-Base-p2-Trash-Large-EXP2")
# Run inference
sentences = [
    'Penjelasan Pasal 37 36',
    'Ayat (4) Cukup jelas. Pasal 37 Cukup jelas.',
    'kawasan wisata yang banyak dikunjungi oleh wisatawan. Jumlah wisatawan yang berkunjung pada saat liburan tahun 2018 mencapai 9.870 orang dalam satu hari. Setiap aktifitas wisatawan akan mengasilkan sampah di kawasan wisata tersebut, terutama sampah organik',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7491, 0.1104],
#         [0.7491, 1.0000, 0.1569],
#         [0.1104, 0.1569, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

  • Size: 7,314 training samples
  • Columns: anchor, positive, and negative
  • Approximate statistics based on the first 1000 samples:
    anchor positive negative
    type string string string
    details
    • min: 3 tokens
    • mean: 9.37 tokens
    • max: 29 tokens
    • min: 11 tokens
    • mean: 45.59 tokens
    • max: 131 tokens
    • min: 5 tokens
    • mean: 40.23 tokens
    • max: 94 tokens
  • Samples:
    anchor positive negative
    kedua Masuk ke Bagian Kedua Bagian Pertamaan administratifphrasing administratif Pasal 75 Ketentuan lebih lanjut mengenai tata cara pengangkatan pejabat pengawas lingkungan hidup dan tata cara pelaksanaan pengawasan sebagaimana dimaksud dalam Pasal 71 ayat (3), Pasal 73, dan Pasal 74 diatur dalam Peraturan Pemerintah. Bagian Kedua - 50 - Bagian Kedua Sanksi Administratif Pasal 76 (1) Menteri, gubernur, atau bupati/walikota menerapkan sanksi administratif kepada penanggung jawab usaha dan/atau kegiatan jika dalam pengawasan ditemukan pelanggaran terhadap izin lingkungan. udara rata-rata adalah 300C. Desa ini berjarak 4 km dari pusat Kecamatan Kretek dan 13 km dari ibukota kabupaten Bantul. Di lingkup wilayah Desa Parangtritis ini daya tarik wisata utama yang
    Apa pidana bagi mereka yang melepaskan produk rekayasa genetik? Kalimat Apa (2) Tindak pidana sebagaimana dimaksud pada ayat (1) hanya dapat dikenakan apabila sanksi administratif yang telah dijatuhkan tidak dipatuhi atau pelanggaran dilakukan lebih dari satu kali. Pasal 101 Setiap orang yang melepaskan dan/atau mengedarkan produk rekayasa genetik ke media lingkungan hidup yang bertentangan dengan peraturan perundang-undangan atau izin lingkungan sebagaimana dimaksud dalam Pasal 69 ayat (1) huruf g, dipidana dengan pidana penjara paling singkat 1 (satu) tahun dan paling lama 3 (tiga) tahun dan denda paling Wisata Pantai di D.I. Yogyakarta sangat banyak, dan selalu bertambah lokasi wisata pantai baru di Yogyakarta. Hal ini, dikarenakan Kelompok Sadar Wisata (Pokdarwis) di Yogyakarta sangat aktif.
    Wewenang penyidik lanjutan melakukan pemeriksaan di tempat tertentu yang diduga terdapat bahan bukti, pembukuan, pencatatan, dan dokumen lain serta melakukan penyitaan terhadap bahan dan barang hasil kejahatan yang dapat dijadikan bukti dalam perkara tindak pidana di bidang pengelolaan sampah; dan f. meminta bantuan ahli dalam pelaksanaan tugas penyidikan tindak pidana di bidang pengelolaan sampah. udara rata-rata adalah 300C. Desa ini berjarak 4 km dari pusat Kecamatan Kretek dan 13 km dari ibukota kabupaten Bantul. Di lingkup wilayah Desa Parangtritis ini daya tarik wisata utama yang
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 64
  • learning_rate: 2e-05
  • fp16: True
  • push_to_hub: True
  • hub_model_id: yosriku/Indobert-Base-p2-Trash-Large-EXP2
  • hub_strategy: end
  • hub_private_repo: False

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: no
  • prediction_loss_only: True
  • per_device_train_batch_size: 64
  • per_device_eval_batch_size: 8
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 3
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.0
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • bf16: False
  • fp16: True
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • project: huggingface
  • trackio_space_id: trackio
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: True
  • resume_from_checkpoint: None
  • hub_model_id: yosriku/Indobert-Base-p2-Trash-Large-EXP2
  • hub_strategy: end
  • hub_private_repo: False
  • hub_always_push: False
  • hub_revision: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: no
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • liger_kernel_config: None
  • eval_use_gather_object: False
  • average_tokens_across_devices: True
  • prompts: None
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: proportional
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Epoch Step Training Loss
0.0870 10 2.5963
0.1739 20 1.6547
0.2609 30 1.4108
0.3478 40 1.1778
0.4348 50 0.9675
0.5217 60 0.9071
0.6087 70 0.7577
0.6957 80 0.794
0.7826 90 0.7271
0.8696 100 0.6572
0.9565 110 0.7096
1.0435 120 0.6268
1.1304 130 0.4045
1.2174 140 0.495
1.3043 150 0.5085
1.3913 160 0.3841
1.4783 170 0.4795
1.5652 180 0.4172
1.6522 190 0.4576
1.7391 200 0.3846
1.8261 210 0.4809
1.9130 220 0.3625
2.0 230 0.4327
2.0870 240 0.348
2.1739 250 0.3248
2.2609 260 0.3245
2.3478 270 0.3752
2.4348 280 0.3184
2.5217 290 0.3321
2.6087 300 0.3317
2.6957 310 0.3084
2.7826 320 0.3074
2.8696 330 0.2833
2.9565 340 0.2959

Framework Versions

  • Python: 3.12.12
  • Sentence Transformers: 5.1.2
  • Transformers: 4.57.2
  • PyTorch: 2.9.0+cu126
  • Accelerate: 1.12.0
  • Datasets: 4.0.0
  • Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
Downloads last month
12
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for yosriku/Indobert-Base-p2-Trash-Large-EXP2

Base model

indobenchmark/indobert-base-p2
Finetuned
(84)
this model