Models
Datasets
Spaces
Docs
Enterprise
Pricing
Log In
Sign Up

KBLab
/

unigram-32k-pretok-small_data-tokenizer

Model card Files Files and versions

unigram-32k-pretok-small_data-tokenizer

2.05 MB

1 contributor

History: 2 commits

robinq's picture

Upload tokenizer

41e3646 about 3 years ago

.gitattributes

1.48 kB

initial commit about 3 years ago
special_tokens_map.json

173 Bytes

Upload tokenizer about 3 years ago
tokenizer.json

2.05 MB

Upload tokenizer about 3 years ago
tokenizer_config.json

322 Bytes

Upload tokenizer about 3 years ago