Add paper abstract and link to model card
#1
by
nielsr
HF Staff
- opened
README.md
CHANGED
|
@@ -1,7 +1,4 @@
|
|
| 1 |
---
|
| 2 |
-
license: mit
|
| 3 |
-
language:
|
| 4 |
-
- en
|
| 5 |
base_model:
|
| 6 |
- distilbert/distilbert-base-uncased
|
| 7 |
datasets:
|
|
@@ -18,7 +15,10 @@ datasets:
|
|
| 18 |
- allenai/qasc
|
| 19 |
- nguyen-brat/worldtree
|
| 20 |
- qiaojin/PubMedQA
|
|
|
|
|
|
|
| 21 |
library_name: transformers
|
|
|
|
| 22 |
tags:
|
| 23 |
- text-classification
|
| 24 |
- sketch-of-thought
|
|
@@ -44,6 +44,7 @@ Unlike conventional Chain of Thought (CoT) approaches that produce verbose reaso
|
|
| 44 |
|
| 45 |
- **Expert Lexicons**: Leverages domain-specific shorthand, technical symbols, and jargon for precise and efficient communication. Suited for technical disciplines requiring maximum information density.
|
| 46 |
|
|
|
|
| 47 |
## Loading the Model
|
| 48 |
|
| 49 |
This repository contains the DistilBERT paradigm selection model for the Sketch-of-Thought (SoT) framework. You can load and use it directly with Hugging Face Transformers:
|
|
@@ -193,6 +194,8 @@ The SoT package supports multiple output formats:
|
|
| 193 |
- `"vlm"`: Multimodal format for vision-language models
|
| 194 |
- `"raw"`: Raw exemplars without formatting
|
| 195 |
|
|
|
|
|
|
|
| 196 |
<details>
|
| 197 |
<summary>What's the difference?</summary>
|
| 198 |
|
|
@@ -268,6 +271,31 @@ The SoT package supports multiple output formats:
|
|
| 268 |
|
| 269 |
SoT supports multiple languages. System prompts and exemplars are automatically loaded in the requested language.
|
| 270 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 271 |
## Limitations
|
| 272 |
|
| 273 |
- The model is trained to classify questions into one of three predefined paradigms and may not generalize to tasks outside the training distribution.
|
|
@@ -285,7 +313,7 @@ If you find our work helpful, please cite:
|
|
| 285 |
eprint={2503.05179},
|
| 286 |
archivePrefix={arXiv},
|
| 287 |
primaryClass={cs.CL},
|
| 288 |
-
url={https://
|
| 289 |
}
|
| 290 |
```
|
| 291 |
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
| 2 |
base_model:
|
| 3 |
- distilbert/distilbert-base-uncased
|
| 4 |
datasets:
|
|
|
|
| 15 |
- allenai/qasc
|
| 16 |
- nguyen-brat/worldtree
|
| 17 |
- qiaojin/PubMedQA
|
| 18 |
+
language:
|
| 19 |
+
- en
|
| 20 |
library_name: transformers
|
| 21 |
+
license: mit
|
| 22 |
tags:
|
| 23 |
- text-classification
|
| 24 |
- sketch-of-thought
|
|
|
|
| 44 |
|
| 45 |
- **Expert Lexicons**: Leverages domain-specific shorthand, technical symbols, and jargon for precise and efficient communication. Suited for technical disciplines requiring maximum information density.
|
| 46 |
|
| 47 |
+
|
| 48 |
## Loading the Model
|
| 49 |
|
| 50 |
This repository contains the DistilBERT paradigm selection model for the Sketch-of-Thought (SoT) framework. You can load and use it directly with Hugging Face Transformers:
|
|
|
|
| 194 |
- `"vlm"`: Multimodal format for vision-language models
|
| 195 |
- `"raw"`: Raw exemplars without formatting
|
| 196 |
|
| 197 |
+
|
| 198 |
+
|
| 199 |
<details>
|
| 200 |
<summary>What's the difference?</summary>
|
| 201 |
|
|
|
|
| 271 |
|
| 272 |
SoT supports multiple languages. System prompts and exemplars are automatically loaded in the requested language.
|
| 273 |
|
| 274 |
+
## Paradigm Selection Model
|
| 275 |
+
|
| 276 |
+
SoT includes a pretrained DistilBERT model for automatic paradigm selection based on the question. The model is available on Hugging Face: [saytes/SoT_DistilBERT](https://huggingface.co/saytes/SoT_DistilBERT)
|
| 277 |
+
|
| 278 |
+
## Datasets
|
| 279 |
+
|
| 280 |
+
The SoT_DistilBERT model was evaluated on the following datasets:
|
| 281 |
+
|
| 282 |
+
| Dataset | HF ID | Subset | Split | Evaluation Type |
|
| 283 |
+
|---------|-------|--------|-------|----------------|
|
| 284 |
+
| GSM8K | [gsm8k](https://huggingface.co/datasets/gsm8k) | main | test | numerical |
|
| 285 |
+
| SVAMP | [ChilleD/SVAMP](https://huggingface.co/datasets/ChilleD/SVAMP) | - | test | numerical |
|
| 286 |
+
| AQUA-RAT | [aqua_rat](https://huggingface.co/datasets/aqua_rat) | - | test | multiple_choice |
|
| 287 |
+
| DROP | [drop](https://huggingface.co/datasets/drop) | - | validation | open |
|
| 288 |
+
| OpenbookQA | [openbookqa](https://huggingface.co/datasets/openbookqa) | - | test | multiple_choice |
|
| 289 |
+
| StrategyQA | [ChilleD/StrategyQA](https://huggingface.co/datasets/ChilleD/StrategyQA) | - | test | yesno |
|
| 290 |
+
| LogiQA | [lucasmccabe/logiqa](https://huggingface.co/datasets/lucasmccabe/logiqa) | default | test | multiple_choice |
|
| 291 |
+
| Reclor | [metaeval/reclor](https://huggingface.co/datasets/metaeval/reclor) | - | validation | multiple_choice |
|
| 292 |
+
| HotPotQA | [hotpot_qa](https://huggingface.co/datasets/hotpot_qa) | distractor | validation | open |
|
| 293 |
+
| MuSiQue-Ans | [dgslibisey/MuSiQue](https://huggingface.co/datasets/dgslibisey/MuSiQue) | - | validation | open |
|
| 294 |
+
| QASC | [allenai/qasc](https://huggingface.co/datasets/allenai/qasc) | - | validation | multiple_choice |
|
| 295 |
+
| Worldtree | [nguyen-brat/worldtree](https://huggingface.co/datasets/nguyen-brat/worldtree) | - | train | multiple_choice |
|
| 296 |
+
| PubMedQA | [qiaojin/PubMedQA](https://huggingface.co/datasets/qiaojin/PubMedQA) | pqa_labeled | train | yesno |
|
| 297 |
+
| MedQA | [bigbio/med_qa](https://huggingface.co/datasets/bigbio/med_qa) | med_qa_en_source | validation | multiple_choice |
|
| 298 |
+
|
| 299 |
## Limitations
|
| 300 |
|
| 301 |
- The model is trained to classify questions into one of three predefined paradigms and may not generalize to tasks outside the training distribution.
|
|
|
|
| 313 |
eprint={2503.05179},
|
| 314 |
archivePrefix={arXiv},
|
| 315 |
primaryClass={cs.CL},
|
| 316 |
+
url={https://hf.co/papers/2503.05179},
|
| 317 |
}
|
| 318 |
```
|
| 319 |
|