google-bert/bert-base-cased
Fill-Mask
•
0.1B
•
Updated
•
4.03M
•
•
321
BERT release
Regroups the original BERT models released by the Google team. Except for the models marked otherwise, the checkpoints support English.
Note Base BERT model, smaller variant. Trained on the "cased" dataset, meaning that it wasn't lowercase and all accents were kept. 12-layer, 768-hidden, 12-heads , 110M parameters
google-bert/bert-base-uncased
Fill-Mask • 0.1B • Updated • 57.6M • • 2.35kNote Base BERT model, smaller variant. Trained on the "uncased" dataset, meaning that it was lowercase and all accents were removed. 12-layer, 768-hidden, 12-heads , 110M parameters
google-bert/bert-large-cased
Fill-Mask • 0.3B • Updated • 172k • • 35Note Large BERT model, larger variant. Trained on the "cased" dataset, meaning that it wasn't lowercase and all accents were kept. 24-layer, 1024-hidden, 16-heads, 340M parameters
google-bert/bert-large-uncased
Fill-Mask • 0.3B • Updated • 842k • • 137Note Large BERT model, larger variant. Trained on the "uncased" dataset, meaning that it was lowercase and all accents were removed. 24-layer, 1024-hidden, 16-heads, 340M parameters
google-bert/bert-base-multilingual-cased
Fill-Mask • 0.2B • Updated • 8.69M • 513Note Base BERT model, smaller variant. The list of supported languages is available here: https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages 104 languages, 12-layer, 768-hidden, 12-heads, 110M parameters
google-bert/bert-base-chinese
Fill-Mask • 0.1B • Updated • 3.45M • • 1.27kNote Base BERT model, smaller variant. Chinese Simplified and Traditional, 12-layer, 768-hidden, 12-heads, 110M parameters
google-bert/bert-large-cased-whole-word-masking
Fill-Mask • 0.3B • Updated • 686 • • 22Note Large BERT model, larger variant. Trained on the "cased" dataset, meaning that it wasn't lowercase and all accents were kept. Whole word masking indicates a different preprocessing where entire words are masked rather than subwords. The BERT team reports better metrics with the wwm models. 24-layer, 1024-hidden, 16-heads, 340M parameters
google-bert/bert-large-uncased-whole-word-masking
Fill-Mask • 0.3B • Updated • 10.2k • 20Note Large BERT model, larger variant. Trained on the "uncased" dataset, meaning that it was lowercase and all accents were removed. Whole word masking indicates a different preprocessing where entire words are masked rather than subwords. The BERT team reports better metrics with the wwm models. 24-layer, 1024-hidden, 16-heads, 340M parameters