README.md

metadata

library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
metrics:
  - accuracy
widget:
  - text: She picks up a wine glass and takes a drink. She
  - text: Someone smiles as she looks out her window. Their car
  - text: >-
      Someone turns and her jaw drops at the site of the other woman. Moving in
      slow motion, someone
  - text: He sneers and winds up with his fist. Someone
  - text: >-
      He smooths it back with his hand. Finally, appearing confident and relaxed
      and with the old familiar glint in his eyes, someone
pipeline_tag: text-classification
inference: true
base_model: sentence-transformers/paraphrase-mpnet-base-v2
model-index:
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.16538461538461538
            name: Accuracy

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A SetFitHead instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
• Classification head: a SetFitHead instance
• Maximum Sequence Length: 512 tokens
• Number of Classes: 9 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
8	'Later she meets someone at the bar. He' 'He heads to them and sits. The bus' 'Someone leaps to his feet and punches the agent in the face. Seemingly unaffected, the agent'
2	'A man sits behind a desk. Two people' 'A man is seen standing at the bottom of a hole while a man records him. Two men' 'Someone questions his female colleague who shrugs. Through a window, we'
0	'A woman bends down and puts something on a scale. She then' 'He pulls down the blind. He' 'Someone flings his hands forward. The someone fires, but the water'
6	'People are sitting down on chairs. They' 'They look up at stained glass skylights. The Americans' 'The lady and the man dance around each other in a circle. The people'
1	'An older gentleman kisses her. As he leads her off, someone' 'The first girl comes back and does it effortlessly as the second girl still struggles. For the last round, the girl' 'As she leaves, the bartender smiles. Now the blonde'
3	'Someone lowers his demoralized gaze. Someone' 'Someone goes into his bedroom. Someone' 'As someone leaves, someone spots him on the monitor. Someone'
7	'Four inches of Plexiglas separate the two and they talk on monitored phones. Someone' 'The American and Russian commanders each watch them returning. As someone' 'A group of walkers walk along the sidewalk near the lake. A man'
4	'The secretary flexes the foot of her crossed - leg as she eyes someone. The woman' 'A man in a white striped shirt is smiling. A woman' 'He grabs her hair and pulls her head back. She'
5	'He heads out of the plaza. Someone' "As he starts back, he sees someone's scared look just before he slams the door shut. Someone" 'He nods at her beaming. Someone'

Evaluation

Metrics

Label	Accuracy
all	0.1654

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("HelgeKn/Swag-multi-class-20")
# Run inference
preds = model("He sneers and winds up with his fist. Someone")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	5	12.1056	33

Label	Training Sample Count
0	20
1	20
2	20
3	20
4	20
5	20
6	20
7	20
8	20

Training Hyperparameters

• batch_size: (16, 16)
• num_epochs: (2, 2)
• max_steps: -1
• sampling_strategy: oversampling
• num_iterations: 20
• body_learning_rate: (2e-05, 2e-05)
• head_learning_rate: 2e-05
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0022	1	0.3747	-
0.1111	50	0.2052	-
0.2222	100	0.1878	-
0.3333	150	0.1126	-
0.4444	200	0.1862	-
0.5556	250	0.1385	-
0.6667	300	0.0154	-
0.7778	350	0.0735	-
0.8889	400	0.0313	-
1.0	450	0.0189	-
1.1111	500	0.0138	-
1.2222	550	0.0046	-
1.3333	600	0.0043	-
1.4444	650	0.0021	-
1.5556	700	0.0033	-
1.6667	750	0.001	-
1.7778	800	0.0026	-
1.8889	850	0.0022	-
2.0	900	0.0014	-

Framework Versions

• Python: 3.9.13
• SetFit: 1.0.1
• Sentence Transformers: 2.2.2
• Transformers: 4.36.0
• PyTorch: 2.1.1+cpu
• Datasets: 2.15.0
• Tokenizers: 0.15.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}