SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A MultiOutputClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Classification head: a MultiOutputClassifier instance
Maximum Sequence Length: 128 tokens

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("faodl/20250909_model_g20_multilabel_MiniLM-L12-all-labels-artificial-governance-multi-output")
# Run inference
preds = model("The program mainly aims at 
the construction of rural roads, capacity building of local bodies, and 
awareness raising activities.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	41.6795	1753

Training Hyperparameters

batch_size: (32, 32)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 50
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0001	1	0.184	-
0.0039	50	0.1927	-
0.0078	100	0.1729	-
0.0117	150	0.1484	-
0.0156	200	0.1301	-
0.0196	250	0.1134	-
0.0235	300	0.1079	-
0.0274	350	0.1021	-
0.0313	400	0.0876	-
0.0352	450	0.0834	-
0.0391	500	0.0886	-
0.0430	550	0.0728	-
0.0469	600	0.0775	-
0.0508	650	0.0811	-
0.0548	700	0.0745	-
0.0587	750	0.0753	-
0.0626	800	0.0745	-
0.0665	850	0.07	-
0.0704	900	0.0702	-
0.0743	950	0.0707	-
0.0782	1000	0.0702	-
0.0821	1050	0.0607	-
0.0860	1100	0.067	-
0.0899	1150	0.065	-
0.0939	1200	0.0659	-
0.0978	1250	0.066	-
0.1017	1300	0.066	-
0.1056	1350	0.06	-
0.1095	1400	0.0609	-
0.1134	1450	0.0587	-
0.1173	1500	0.0542	-
0.1212	1550	0.0523	-
0.1251	1600	0.0559	-
0.1291	1650	0.052	-
0.1330	1700	0.0487	-
0.1369	1750	0.053	-
0.1408	1800	0.0477	-
0.1447	1850	0.0492	-
0.1486	1900	0.0474	-
0.1525	1950	0.0488	-
0.1564	2000	0.0461	-
0.1603	2050	0.0481	-
0.1643	2100	0.0463	-
0.1682	2150	0.0432	-
0.1721	2200	0.0482	-
0.1760	2250	0.0444	-
0.1799	2300	0.0466	-
0.1838	2350	0.0423	-
0.1877	2400	0.041	-
0.1916	2450	0.0422	-
0.1955	2500	0.0401	-
0.1995	2550	0.0405	-
0.2034	2600	0.0448	-
0.2073	2650	0.0387	-
0.2112	2700	0.0371	-
0.2151	2750	0.0429	-
0.2190	2800	0.0379	-
0.2229	2850	0.0384	-
0.2268	2900	0.0378	-
0.2307	2950	0.0392	-
0.2346	3000	0.038	-
0.2386	3050	0.0325	-
0.2425	3100	0.0345	-
0.2464	3150	0.0341	-
0.2503	3200	0.0415	-
0.2542	3250	0.0313	-
0.2581	3300	0.0355	-
0.2620	3350	0.033	-
0.2659	3400	0.0308	-
0.2698	3450	0.0343	-
0.2738	3500	0.0379	-
0.2777	3550	0.032	-
0.2816	3600	0.0358	-
0.2855	3650	0.0334	-
0.2894	3700	0.0312	-
0.2933	3750	0.0336	-
0.2972	3800	0.0291	-
0.3011	3850	0.0268	-
0.3050	3900	0.034	-
0.3090	3950	0.0337	-
0.3129	4000	0.0266	-
0.3168	4050	0.0269	-
0.3207	4100	0.0326	-
0.3246	4150	0.0317	-
0.3285	4200	0.0271	-
0.3324	4250	0.0313	-
0.3363	4300	0.0263	-
0.3402	4350	0.0267	-
0.3442	4400	0.0273	-
0.3481	4450	0.026	-
0.3520	4500	0.0252	-
0.3559	4550	0.0261	-
0.3598	4600	0.0243	-
0.3637	4650	0.0252	-
0.3676	4700	0.0291	-
0.3715	4750	0.0286	-
0.3754	4800	0.0245	-
0.3794	4850	0.0263	-
0.3833	4900	0.0249	-
0.3872	4950	0.0209	-
0.3911	5000	0.0245	-
0.3950	5050	0.0278	-
0.3989	5100	0.0277	-
0.4028	5150	0.0266	-
0.4067	5200	0.0249	-
0.4106	5250	0.0279	-
0.4145	5300	0.027	-
0.4185	5350	0.0283	-
0.4224	5400	0.022	-
0.4263	5450	0.0232	-
0.4302	5500	0.0198	-
0.4341	5550	0.0254	-
0.4380	5600	0.0186	-
0.4419	5650	0.0237	-
0.4458	5700	0.0249	-
0.4497	5750	0.0241	-
0.4537	5800	0.0239	-
0.4576	5850	0.0258	-
0.4615	5900	0.0212	-
0.4654	5950	0.0208	-
0.4693	6000	0.0227	-
0.4732	6050	0.0262	-
0.4771	6100	0.0257	-
0.4810	6150	0.0227	-
0.4849	6200	0.0226	-
0.4889	6250	0.0231	-
0.4928	6300	0.0255	-
0.4967	6350	0.0199	-
0.5006	6400	0.022	-
0.5045	6450	0.0253	-
0.5084	6500	0.0209	-
0.5123	6550	0.0207	-
0.5162	6600	0.0215	-
0.5201	6650	0.0225	-
0.5241	6700	0.0185	-
0.5280	6750	0.019	-
0.5319	6800	0.0214	-
0.5358	6850	0.0252	-
0.5397	6900	0.0216	-
0.5436	6950	0.0205	-
0.5475	7000	0.0205	-
0.5514	7050	0.0244	-
0.5553	7100	0.0223	-
0.5592	7150	0.0181	-
0.5632	7200	0.0199	-
0.5671	7250	0.0217	-
0.5710	7300	0.0198	-
0.5749	7350	0.0224	-
0.5788	7400	0.0234	-
0.5827	7450	0.0193	-
0.5866	7500	0.0168	-
0.5905	7550	0.0193	-
0.5944	7600	0.0232	-
0.5984	7650	0.0183	-
0.6023	7700	0.0255	-
0.6062	7750	0.0209	-
0.6101	7800	0.0262	-
0.6140	7850	0.0228	-
0.6179	7900	0.0208	-
0.6218	7950	0.0167	-
0.6257	8000	0.0217	-
0.6296	8050	0.0175	-
0.6336	8100	0.0196	-
0.6375	8150	0.0215	-
0.6414	8200	0.0186	-
0.6453	8250	0.0181	-
0.6492	8300	0.0171	-
0.6531	8350	0.0224	-
0.6570	8400	0.0214	-
0.6609	8450	0.0214	-
0.6648	8500	0.0192	-
0.6688	8550	0.0213	-
0.6727	8600	0.0185	-
0.6766	8650	0.02	-
0.6805	8700	0.0218	-
0.6844	8750	0.0163	-
0.6883	8800	0.0183	-
0.6922	8850	0.0177	-
0.6961	8900	0.0178	-
0.7000	8950	0.0157	-
0.7039	9000	0.0201	-
0.7079	9050	0.017	-
0.7118	9100	0.0198	-
0.7157	9150	0.0196	-
0.7196	9200	0.0189	-
0.7235	9250	0.018	-
0.7274	9300	0.0193	-
0.7313	9350	0.0179	-
0.7352	9400	0.0218	-
0.7391	9450	0.0186	-
0.7431	9500	0.0175	-
0.7470	9550	0.0168	-
0.7509	9600	0.0193	-
0.7548	9650	0.0183	-
0.7587	9700	0.0168	-
0.7626	9750	0.0194	-
0.7665	9800	0.021	-
0.7704	9850	0.0178	-
0.7743	9900	0.018	-
0.7783	9950	0.0171	-
0.7822	10000	0.0191	-
0.7861	10050	0.0147	-
0.7900	10100	0.0193	-
0.7939	10150	0.0174	-
0.7978	10200	0.0171	-
0.8017	10250	0.0156	-
0.8056	10300	0.0176	-
0.8095	10350	0.0195	-
0.8135	10400	0.0151	-
0.8174	10450	0.0192	-
0.8213	10500	0.0201	-
0.8252	10550	0.0192	-
0.8291	10600	0.015	-
0.8330	10650	0.0181	-
0.8369	10700	0.0143	-
0.8408	10750	0.0177	-
0.8447	10800	0.015	-
0.8487	10850	0.0193	-
0.8526	10900	0.0168	-
0.8565	10950	0.0169	-
0.8604	11000	0.0166	-
0.8643	11050	0.0148	-
0.8682	11100	0.0163	-
0.8721	11150	0.0189	-
0.8760	11200	0.0197	-
0.8799	11250	0.0138	-
0.8838	11300	0.0168	-
0.8878	11350	0.0153	-
0.8917	11400	0.0147	-
0.8956	11450	0.0178	-
0.8995	11500	0.0184	-
0.9034	11550	0.0158	-
0.9073	11600	0.0183	-
0.9112	11650	0.0127	-
0.9151	11700	0.0169	-
0.9190	11750	0.018	-
0.9230	11800	0.0156	-
0.9269	11850	0.0156	-
0.9308	11900	0.0162	-
0.9347	11950	0.0124	-
0.9386	12000	0.0175	-
0.9425	12050	0.0179	-
0.9464	12100	0.0182	-
0.9503	12150	0.0176	-
0.9542	12200	0.0182	-
0.9582	12250	0.0189	-
0.9621	12300	0.0125	-
0.9660	12350	0.0176	-
0.9699	12400	0.0143	-
0.9738	12450	0.0162	-
0.9777	12500	0.017	-
0.9816	12550	0.0196	-
0.9855	12600	0.0192	-
0.9894	12650	0.0184	-
0.9934	12700	0.0149	-
0.9973	12750	0.0172	-

Framework Versions

Python: 3.12.11
SetFit: 1.1.3
Sentence Transformers: 5.1.0
Transformers: 4.56.1
PyTorch: 2.8.0+cu126
Datasets: 4.0.0
Tokenizers: 0.22.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

Downloads last month: 252

Safetensors

Model size

0.1B params

Tensor type

F32

Model tree for faodl/20250909_model_g20_multilabel_MiniLM-L12-all-labels-artificial-governance-multi-output

Base model

sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

Finetuned

(257)

this model