clemsail/micro-kiki-v4-sota

Micro-Kiki 35B A3B V4-SOTA LoRA Adapters

Collection of 35 domain-specialized LoRA adapters trained on top of Qwen3.6-35B-A3B (MoE 35B, 8 experts of 256, A3B activation).

Produced as part of the micro-kiki project, L'Electron Rare, April 2026.

Training setup

Parameter	Value
Framework	mlx_lm (Apple Silicon MLX)
Hardware	Mac Studio M3 Ultra, 512 GB unified memory
Fine-tune type	LoRA
Rank	16
Alpha	16
Dropout	0.0
Scale	20.0
Target layers	32
Iterations	200 per domain
Batch size	1
Learning rate	1e-5
Max seq length	1024
Grad checkpoint	true

Domains (35)

chat-fr, components, cpp, devops, docker, dsp, electronics, embedded, emc, freecad, html-css, iot, kicad-dsl, kicad-pcb, llm-ops, llm-orch, lua-upy, math, ml-training, music-audio, platformio, power, python, reasoning, rust, security, shell, spice, spice-sim, sql, stm32, typescript, web-backend, web-frontend, yaml-json.

Repository layout

<domain>/
  adapters.safetensors             # final LoRA weights
  0000200_adapters.safetensors     # checkpoint at iter 200
  adapter_config.json              # PEFT config
config-<domain>.yaml               # training config per domain
log-<domain>.txt                   # training log (loss curves, warnings)

Usage (MLX)

from mlx_lm import load, generate

model, tokenizer = load(
    "Qwen/Qwen3.6-35B-A3B",
    adapter_path="path/to/this-repo/math",
)
print(generate(model, tokenizer, prompt="Prove the Pythagorean theorem.", max_tokens=512))

Notes

• Sequences longer than 1024 tokens were truncated during training.
• llm-ops domain had a Metal backend crash right after save; weights are intact but the post-training eval did not complete.
• Four domains (components, electronics, llm-ops, security) are ~2.4 GB instead of 3.8 GB — smaller datasets or shorter effective rank.

Citation

If you use these adapters, please cite:

@software{microkiki_v4sota_2026,
  author  = {Saillant, Clément},
  title   = {Micro-Kiki 35B A3B V4-SOTA LoRA Adapters},
  year    = {2026},
  month   = {4},
  url     = {https://huggingface.co/electron-rare/micro-kiki-35b-a3b-v4-sota-lora}
}