README.md

---
datasets:
- HuggingFaceH4/ultrachat_200k
base_model:
- meta-llama/Llama-3.2-1B-Instruct
library_name: transformers
---

## meta-llama/Llama-3.2-1B-Instruct - W8A8_int8 Compression

This is a compressed model using [llmcompressor](https://github.com/vllm-project/llm-compressor).

## Compression Configuration

- Base Model: meta-llama/Llama-3.2-1B-Instruct
- Compression Scheme: W8A8_int8
- Dataset: HuggingFaceH4/ultrachat_200k
- Dataset Split: train_sft
- Number of Samples: 512
- Preprocessor: chat
- Maximum Sequence Length: 131072

## Sample Output

#### Prompt: 

```
<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 28 Nov 2024

<|eot_id|><|start_header_id|>user<|end_header_id|>

Who is Alan Turing?<|eot_id|>
```

#### Output: 

```
<|begin_of_text|><|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 28 Nov 2024

<|eot_id|><|start_header_id|>user<|end_header_id|>

Who is Alan Turing?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Alan Turing (1912-1954) was a British mathematician, computer scientist, logician, and philosopher who made significant contributions to the development of computer science, artificial intelligence, and cryptography. He is widely regarded as one of the most influential figures of the 20th century.

Turing was born in London, England, to a family of mathematicians and engineers. He showed an early interest in mathematics and programming, and was a brilliant student at King's College, Cambridge, where he worked on the theoretical foundations of computation.

In 1936, Turing was recruited by the British government to work on the
```

## Evaluation

<TODO>