Update README.md
Browse files
README.md
CHANGED
|
@@ -68,10 +68,6 @@ text = tokenizer.apply_chat_template(
|
|
| 68 |
add_generation_prompt=True
|
| 69 |
)
|
| 70 |
|
| 71 |
-
# avoid empty thought content by forcing the model to start with "<think>\n"
|
| 72 |
-
response_prefix = "<think>\n"
|
| 73 |
-
text += response_prefix
|
| 74 |
-
|
| 75 |
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
| 76 |
|
| 77 |
generated_ids = model.generate(
|
|
@@ -83,14 +79,14 @@ generated_ids = [
|
|
| 83 |
]
|
| 84 |
|
| 85 |
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
| 86 |
-
print(
|
| 87 |
```
|
| 88 |
|
| 89 |
### Usage Guidelines
|
| 90 |
|
| 91 |
To achieve optimal performance, we recommend the following settings:
|
| 92 |
|
| 93 |
-
1. **Enforce Thoughtful Output**: Ensure the model starts with "\<think\>\n" to prevent generating empty thinking content, which can degrade output quality.
|
| 94 |
|
| 95 |
2. **Sampling Parameters**:
|
| 96 |
- Use Temperature=0.6 and TopP=0.95 instead of Greedy decoding to avoid endless repetitions and enhance diversity.
|
|
|
|
| 68 |
add_generation_prompt=True
|
| 69 |
)
|
| 70 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
| 72 |
|
| 73 |
generated_ids = model.generate(
|
|
|
|
| 79 |
]
|
| 80 |
|
| 81 |
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
| 82 |
+
print(response)
|
| 83 |
```
|
| 84 |
|
| 85 |
### Usage Guidelines
|
| 86 |
|
| 87 |
To achieve optimal performance, we recommend the following settings:
|
| 88 |
|
| 89 |
+
1. **Enforce Thoughtful Output**: Ensure the model starts with "\<think\>\n" to prevent generating empty thinking content, which can degrade output quality. If you use `apply_chat_template` and set `add_generation_prompt=True`, this is already automatically implemented, but it may cause the response to lack the \<think\> tag at the beginning. This is normal behavior.
|
| 90 |
|
| 91 |
2. **Sampling Parameters**:
|
| 92 |
- Use Temperature=0.6 and TopP=0.95 instead of Greedy decoding to avoid endless repetitions and enhance diversity.
|