Instructions to use lianghsun/Llama-3.2-Taiwan-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lianghsun/Llama-3.2-Taiwan-1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lianghsun/Llama-3.2-Taiwan-1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lianghsun/Llama-3.2-Taiwan-1B")
model = AutoModelForCausalLM.from_pretrained("lianghsun/Llama-3.2-Taiwan-1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use lianghsun/Llama-3.2-Taiwan-1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lianghsun/Llama-3.2-Taiwan-1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.2-Taiwan-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lianghsun/Llama-3.2-Taiwan-1B

SGLang

How to use lianghsun/Llama-3.2-Taiwan-1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lianghsun/Llama-3.2-Taiwan-1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.2-Taiwan-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lianghsun/Llama-3.2-Taiwan-1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.2-Taiwan-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lianghsun/Llama-3.2-Taiwan-1B with Docker Model Runner:
```
docker model run hf.co/lianghsun/Llama-3.2-Taiwan-1B
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Model Card for Llama-3.2-Taiwan-1B

Llama-3.2-Taiwan-1B 是以 meta-llama/Llama-3.2-1B 為基底，針對繁體中文與中華民國台灣語境完成持續預訓練（continued pretraining, CPT）的小型基底模型，作為 Llama-3.2-Taiwan-1B-Instruct 等下游模型的繁中底座。

⚠️ 規格重點：本模型為 1B 參數小型語言模型（SLM）、純文本單模態、僅做 CPT、未做指令微調，需自行 SFT 後才有對話能力。

Model Details

Meta 釋出的 Llama-3.2 1B 是端側部署的良好候選，但其原生繁中能力有限、對台灣本地語境覆蓋不足。本模型透過大規模繁中持續預訓練（涵蓋網頁、教育、生活、政府公開文本等多類別語料），把繁中底層能力與台灣語境注入模型參數，作為下游 SFT／DPO 的起點。

核心特點 (Key Features)

小型化、部署成本低：1B 參數，可在消費級 GPU、Apple Silicon、邊緣裝置上順暢執行。
繁中底座：訓練語料以繁體中文與台灣本地語境為主，補強原版 Llama-3.2-1B 在繁中流暢度與在地語境理解上的不足。
下游可微調：作為 instruct、領域應用、persona 微調的起點。

Model Description

Developed by: Liang Hsun Huang
Funded by: APMIC
Base model: meta-llama/Llama-3.2-1B
Model type: LlamaForCausalLM (Transformers)
Language(s) (NLP): Traditional Chinese, English
License: llama3.2

Model Sources

Repository: lianghsun/Llama-3.2-Taiwan-1B

Citation

@misc{llama_3_2_taiwan_1b,
  title        = {Llama-3.2-Taiwan-1B: A Traditional Chinese Continued-Pretrained Llama-3.2 1B Model for Taiwan},
  author       = {Huang, Liang Hsun},
  year         = {2024},
  howpublished = {\url{https://huggingface.co/lianghsun/Llama-3.2-Taiwan-1B}}
}