sieve-llama-3.2-1b

Fine-tune of meta-llama/Llama-3.2-1B for SQL WHERE clause generation.

GGUF quantizations are available at azatvaliev/sieve-llama-3.2-1b-GGUF.

Overview

This is a completion model (not instruct/chat). Given a PostgreSQL schema (DDL), a natural language filter as a -- filter: comment, and a SELECT * FROM table prefix, the model completes with the appropriate WHERE clause.

Example

Input:

CREATE TABLE "public"."product" (
  "id" bigint PRIMARY KEY,
  "name" text NOT NULL,
  "category" text NOT NULL,
  "price" numeric(10,2) NOT NULL,
  "in_stock" boolean NOT NULL DEFAULT true
);
-- filter: electronics under $50
SELECT * FROM product 

Output:

WHERE category LIKE '%electronics%' AND price < 50

Training

• Method: LoRA (r=16, alpha=32, dropout=0.10) via TRL's SFTTrainer
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Dataset: 500 examples (completion-only loss)
• Epochs: 3
• Hardware: H100
• Precision: bf16

The adapter/ directory contains the LoRA adapter weights.