| --- |
| license: mit |
| base_model: JackFram/llama-68m |
| tags: |
| - tiny-model |
| - random-weights |
| - testing |
| - llama |
| --- |
| |
| # Llama-3.3-Tiny-Instruct |
|
|
| This is a tiny random version of the JackFram/llama-68m model, created for testing and experimentation purposes. |
|
|
| ## Model Details |
|
|
| - **Base model**: JackFram/llama-68m |
| - **Seed**: 42 |
| - **Hidden size**: 768 |
| - **Number of layers**: 2 |
| - **Number of attention heads**: 12 |
| - **Vocabulary size**: 32000 |
| - **Max position embeddings**: 2048 |
|
|
| ## Parameters |
|
|
| - **Total parameters**: ~43,454,976 |
| - **Trainable parameters**: ~43,454,976 |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForSequenceClassification, AutoTokenizer |
| |
| # Load model and tokenizer |
| model = AutoModelForSequenceClassification.from_pretrained("AlignmentResearch/Llama-3.3-Tiny-Classifier") |
| tokenizer = AutoTokenizer.from_pretrained("AlignmentResearch/Llama-3.3-Tiny-Classifier") |
| |
| # Generate text (note: this model has random weights!) |
| inputs = tokenizer("Hello, how are you?", return_tensors="pt") |
| outputs = model.generate(**inputs, max_length=50) |
| print(tokenizer.decode(outputs[0])) |
| ``` |
|
|
| ## Important Notes |
|
|
| ⚠️ **This model has random weights and is not trained!** It's designed for: |
| - Testing model loading and inference pipelines |
| - Benchmarking model architecture |
| - Educational purposes |
| - Rapid prototyping where actual model performance isn't needed |
|
|
| The model will generate random/nonsensical text since it hasn't been trained on any data. |
|
|
| ## Creation |
|
|
| This model was created using the `upload_tiny_llama33.py` script from the minimal-grpo-trainer repository. |
|
|
