File size: 1,033 Bytes
1df96ff 45ac32f 1df96ff 45ac32f 1df96ff 45ac32f 1df96ff 45ac32f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
---
version: main
family: smollm2-1.7b
model_name: all_raw_folders_metadata-600B
license: mit
tags:
- model
- transformer
- smollm2
---
# SmolLM2 all_raw_folders_metadata-600B (Version: main)
## Model Details
- **Architecture:** SmolLM2
- **Parameters:** 1.7B
## Training Configuration
```yaml
optimizer:
class_path: torch.optim.AdamW
init_args:
lr: 0.0005
weight_decay: 0.01
precision: bf16-mixed
seed: 42
train:
global_batch_size: 1024
max_seq_length: 2048
max_tokens: 600000000000
micro_batch_size: 8
```
## Model Loading and Revision System
This repository hosts multiple revisions of the model.
To load a specific revision, use the `revision` parameter. For example:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("locuslab/all_raw_folders_metadata-600B", revision="final")
tokenizer = AutoTokenizer.from_pretrained("locuslab/all_raw_folders_metadata-600B", revision="final")
```
Replace `"final"` with the desired revision.
|