ziksy/dataset-templater-qwen35-lora

Dataset Templater LoRA — Qwen3.5-27B

A LoRA adapter that teaches Qwen3.5-27B to produce JSON templates mapping arbitrary HuggingFace datasets into ChatML QA training pairs, and to refuse datasets whose rows carry no usable QA content (embeddings, numeric telemetry, bibliographic metadata).

Input / Output

Input: column names + 1–3 sample rows from a HuggingFace dataset.

Output: a single-line JSON object, one of two shapes:

{"question": "{instruction}\n{input}", "answer": "{output}"}

or, for unusable shapes:

{"usable": false, "reason": "rows are embeddings, not QA pairs"}

Benchmarks

197-item held-out test set; 27% hard negatives across varied shapes (five-axis rubric).

Axis	Base 27B	Base + LoRA	Delta
valid_json	100.0%	100.0%	=
correct_refuse	14.7%	18.8%	+4.1
column_sanity	68.5%	91.9%	+23.4
shape_match	50.8%	77.2%	+26.4
exact	38.6%	69.5%	+30.9

Full per-family breakdown and per-example results are in eval/lora_results.jsonl and adapters/qwen35-27b/scores.json.

Training

• Base model: Qwen/Qwen3.5-27B
• Examples: 2,000 (synthetic corpus, 27% hard negatives with variety)
• Rank: 16, Alpha: 32, Epochs: 3
• LR: 2e-5, Batch: 4 × grad_accum 4, BF16
• Hardware: A100 SXM4 80GB (Vast.ai), 2h11m training
• Final train_loss: 0.6338 · eval_loss: 0.311

Known Limitations

• neg_tabular (weather-style numeric rows with innocuous column names like temperature_c, humidity_pct, pressure_hpa): 0/10 refuse. LoRA confidently templates these as QA even though rows are numeric telemetry. This failure mode is stylistic — the corpus-v2 rewrite added many other tabular shapes to teach refusal and those succeed (neg_tabular_v2 9/10), but the specific weather vocabulary wasn't rebalanced enough.
• legal_structured (Clarus-style adversarial case): 0/17 shape_match. The LoRA identifies the input_* fields (17/17 column_sanity) but pairs them differently than ground truth. Base 27B also gets 0/17 — neither scale nor this LoRA solves this specific trap yet.

Usage

Hot-load on llama-server

# Start llama-server with Qwen3.5-27B and this adapter
llama-server \
  -m Qwen3.5-27B-Q4_K_M.gguf \
  --lora dataset-templater-qwen35-lora-f16.gguf \
  --host 127.0.0.1 --port 8788 \
  --reasoning off --jinja -ngl 99

Files

• adapters/qwen35-27b/dataset-templater-qwen35-lora-f16.gguf — the adapter (152 MB F16)
• adapters/qwen35-27b/training_stats.json — loss, hyperparameters
• adapters/qwen35-27b/scores.json — full eval numbers
• agora-manifest.json — discovery manifest (Agora registry format)
• eval/questions.jsonl — 197-item held-out test set
• eval/lora_results.jsonl — per-example predictions + scores
• train_lora.py · eval_remote.py — reproducibility scripts