Per Egil Kummervold
commited on
Commit
·
911bf45
1
Parent(s):
46f4e92
updated with single tokens
Browse files- .README.md.un~ +0 -0
- README.md +0 -9
.README.md.un~
CHANGED
|
Binary files a/.README.md.un~ and b/.README.md.un~ differ
|
|
|
README.md
CHANGED
|
@@ -43,15 +43,6 @@ The following tokens are replaced:
|
|
| 43 |
</tbody>
|
| 44 |
</table>
|
| 45 |
|
| 46 |
-
|
| 47 |
-
| Token ID | Token |
|
| 48 |
-
|----------|---------------------|
|
| 49 |
-
| 128013 | <|think|> |
|
| 50 |
-
| 128014 | `<|/think|>` |
|
| 51 |
-
| 128015 | `<|answer|>` |
|
| 52 |
-
| 128016 | `<|/answer|>` |
|
| 53 |
-
|
| 54 |
-
|
| 55 |
The script (`replace_reserved_tokens.py`) for changing the tokens are also included in this repo.
|
| 56 |
|
| 57 |
# Verification
|
|
|
|
| 43 |
</tbody>
|
| 44 |
</table>
|
| 45 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
The script (`replace_reserved_tokens.py`) for changing the tokens are also included in this repo.
|
| 47 |
|
| 48 |
# Verification
|