Japanese‑language BERT‑Base (12 layers, 768‑dim, 12 heads) pretrained with Unidic‑based word‑level plus character‑level tokenization and whole‑word masking on CC‑100 and 2023 Wikipedia, producing a 7,027‑token vocabulary.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
130,000
Hugging Face likes
11
Listed
May 13, 2024
Downloads (30-day window) over time
Show as a table
| Month | DL (30d) |
|---|---|
| Dec 2025 | 108,000 |
| Jan 2026 | 97,000 |
| Feb 2026 | 97,000 |
| Mar 2026 | 95,000 |
| Apr 2026 | 118,000 |
| May 2026 | 128,000 |
| Jun 2026 | 124,000 |
| Jul 2026 | 121,000 |
| Aug 2026 | 132,000 |
| Sep 2026 | 121,000 |
| Oct 2026 | 122,000 |
| Nov 2026 | 130,000 |
Downloads over the 30 days before the last snapshot of each month.
Popular in the same category
- bert-large-japanese-v2tohoku-nlp66kJapanese‑BERT‑Large trained on CC‑100 and Wikipedia, using Unidic‑lite word‑level tokenization with WordPiece subwords and whole‑word masking (24 layers, 1024‑dim hidden, 16 heads, 32k vocab), with pretraining code on cl‑tohoku/bert‑japanese.
- bert-base-japanese-v3tohoku-nlp66kJapanese BERT‑base (12 layers, 768‑dim hidden, 12 heads, 32 k vocab) pretrained with whole‑word masking on CC‑100 and 2023‑Jan Wikipedia, using Unidic 2.1.2 word‑level tokenization plus WordPiece, in 2 M training steps.
- t5-base-japanese-v1.1sonoisa17kA Japanese T5‑v1.1 model pretrained on ≈100 GB of Wikipedia and OSCAR CC‑100 data (mixed 10:1 for SentencePiece with byte‑fallback), requiring fine‑tuning for downstream tasks, includes transfer‑learning sample code, notes potential bias in outputs, and is licensed CC‑BY‑SA 4.0.
- deberta-v3-base-japaneseku-nlp13kJapanese DeBERTa V3 base pre‑trained on 540 B tokens from LLM‑jp v1.0, trained with a modified DeBERTa V3 setup, uses a unigram byte‑fallback tokenizer (no morphological analyzer), and is fine‑tuned for JGLUE NLU tasks.
- t5-small-shortretrieva-jp3kA T5 v1.1 Japanese‑language model pre‑trained on mC4/ja and Wikipedia, featuring GEGLU activations, no dropout during pre‑training, separate embedding/classifier layers, and larger d_model with fewer heads, licensed CC‑BY‑SA 4.0 for commercial use.
- bert-large-japanese-char-v2tohoku-nlp2kA Japanese‑language BERT large model (24 layers, 1024 hidden size, 16 heads) pretrained on CC‑100 and Wikipedia using Unidic 2.1.2 word‑level tokenization followed by character splitting, with whole‑word masking for MLM, available via cl-tohoku/bert-japanese.
Resources in similar research fields
- bert-base-japanese-whole-word-maskingtohoku-nlp462kJapanese BERT‑base pretrained on 2019 Japanese Wikipedia with IPA‑dictionary and whole‑word masking, 12‑layer 768‑dim, 32,000‑vocab, 512‑token sequences, 1 M steps, available at cl‑tohoku/bert‑japanese under CC‑BY‑SA.Shared: Representation Learning・Syntactic Text Processing
- bert-base-japanese-char-v2tohoku-nlp55kA BERT‑base Japanese model (12 layers, 768‑dim hidden states, 12 heads) trained on 30 M Wikipedia sentences (~4 GB) with Unidic 2.1.2 word‑level tokenization followed by character‑level tokenization and whole‑word masking, using 512‑token sequences, 256 batches, and 1 M training steps.Shared: Representation Learning・Syntactic Text Processing
- bert-base-japanese-chartohoku-nlp11kA BERT‑base Japanese model (12 layers, 768‑dim hidden, 12 heads) pretrained on ~17 M sentences from Japanese Wikipedia (2.6 GB) using MeCab IPA word‑level tokenization followed by character tokenization into a 4000‑word vocabulary, with training code at cl‑tohoku/bert‑japanese and released under CC BY‑SA 3.0.Shared: Representation Learning・Syntactic Text Processing
- bert-base-japanese-v2tohoku-nlp9kJapanese BERT‑base (12 layers, 768 hidden, 12 heads) pretrained on 4 GB of Japanese Wikipedia (≈30 M sentences) with Unidic 2.1.2 word‑level tokenization, WordPiece sub‑tokenization, and whole‑word masking.Shared: Representation Learning・Syntactic Text Processing
- bert-base-japanesetohoku-nlp9kA BERT base model pretrained on ~17 M Japanese Wikipedia sentences (2.6 GB) that tokenizes with the IPA dictionary and WordPiece, has 12 layers/768‑dim hidden states/12 heads, a 32 000‑token vocabulary, was trained for 1 M steps on Cloud TPUs and is released under CC‑BY‑SA 3.0.Shared: Representation Learning・Syntactic Text Processing
- deberta-v2-large-japanese-char-wwmku-nlp76kJapanese DeBERTa V2 large model trained on 171 GB of Japanese Wikipedia, CC‑100, and OSCAR with character‑level sentencepiece tokenization and whole‑word masking, ready for downstream fine‑tuning through Hugging Face Transformers.Shared: Syntactic Text Processing・Text Segmentation