tiodh/llama3.1-8b-jssp-rslora

Llama-3.1-8B-Instruct + rsLoRA — Job-Shop Scheduling

A rsLoRA adapter fine-tuned on the Starjob job-shop scheduling problem (JSSP) dataset. The model takes a natural-language description of jobs and machines and produces a feasible schedule that minimizes makespan.

Training

Hyperparameter	Value
Method	rsLoRA (use_rslora = true)
LoRA rank r	32
LoRA alpha	32
Max sequence length	8192
Per-device batch	1
Gradient accumulation	8 (effective batch 8)
Epochs	1
Learning rate	2e-4
Base quantization	bnb 4-bit (Unsloth)

Evaluation

200 samples (seed 42) from the small+medium split of Starjob, identical pipeline for LoRA and rsLoRA. Feasibility validates routing order, machine non-overlap, and operation completeness.

Metric	Value
Feasibility	95.0% (190/200)
Exact makespan	32.0% (64/200)
Mean gap	9.80%
Median gap	5.29%
Eval time	24.2 min

Full head-to-head LoRA vs rsLoRA comparison and code: github.com/tiodh/slm_jssp.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Meta-Llama-3.1-8B-Instruct",
    device_map="auto",
    torch_dtype="auto",
)
tok = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
model = PeftModel.from_pretrained(base, "tiodh/llama3.1-8b-jssp-rslora")

prompt = (
    "Optimize schedule for 3 Jobs (denoted as J) across 3 Machines (denoted as M) "
    "to minimize makespan...\nJ0:\nM0:5 M1:3 M2:4\nJ1:\nM1:2 M0:4 M2:3\nJ2:\nM2:6 M0:1 M1:5\n"
)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.1, top_p=0.95)
print(tok.decode(out[0], skip_special_tokens=True))

License

CC BY-SA 4.0 (inherits from the Starjob dataset).