|
|
--- |
|
|
language: |
|
|
- ko |
|
|
base_model: |
|
|
- EleutherAI/polyglot-ko-1.3b |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- ingredient |
|
|
- chatbot |
|
|
- review |
|
|
- usage |
|
|
license: apache-2.0 |
|
|
--- |
|
|
|
|
|
## 🐥 Base Model |
|
|
|
|
|
**EleutherAI/polyglot-ko-1.3b** |
|
|
|
|
|
This model is based on the Korean version of Polyglot-1.3B, an open-source language model released by EleutherAI. |
|
|
It is pre-trained on a large-scale Korean corpus and designed for general-purpose Korean language understanding and generation tasks. |
|
|
|
|
|
--- |
|
|
|
|
|
## Training Procedure |
|
|
|
|
|
### Training Hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
|
|
|
- `output_dir`: `./qlora_model_eleutherai` |
|
|
- `per_device_train_batch_size`: `2` |
|
|
- `gradient_accumulation_steps`: `4` |
|
|
- `total_batch_size`: `8 (2 x 4)` |
|
|
- `learning_rate`: `2e-5` |
|
|
- `num_train_epochs`: `2` |
|
|
- `fp16`: `True` |
|
|
- `logging_dir`: `./logs` |
|
|
- `logging_steps`: `5` |
|
|
- `save_steps`: `100` |
|
|
- `save_total_limit`: `1` |
|
|
- `load_best_model_at_end`: `True` |
|
|
- `metric_for_best_model`: `loss` |
