Japanese‑BERT‑Large trained on CC‑100 and Wikipedia, using Unidic‑lite word‑level tokenization with WordPiece subwords and whole‑word masking (24 layers, 1024‑dim hidden, 16 heads, 32k vocab), with pretraining code on cl‑tohoku/bert‑japanese.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
66,000
Hugging Face likes
14
Listed
May 13, 2024
Downloads (30-day window) over time
Show as a table
| Month | DL (30d) |
|---|---|
| Dec 2025 | 69,000 |
| Jan 2026 | 68,000 |
| Feb 2026 | 65,000 |
| Mar 2026 | 65,000 |
| Apr 2026 | 68,000 |
| May 2026 | 72,000 |
| Jun 2026 | 68,000 |
| Jul 2026 | 69,000 |
| Aug 2026 | 80,000 |
| Sep 2026 | 69,000 |
| Oct 2026 | 65,000 |
| Nov 2026 | 66,000 |
Downloads over the 30 days before the last snapshot of each month.
Popular in the same category
- bert-base-japanese-char-v3tohoku-nlp130kJapanese‑language BERT‑Base (12 layers, 768‑dim, 12 heads) pretrained with Unidic‑based word‑level plus character‑level tokenization and whole‑word masking on CC‑100 and 2023 Wikipedia, producing a 7,027‑token vocabulary.
- bert-base-japanese-v3tohoku-nlp66kJapanese BERT‑base (12 layers, 768‑dim hidden, 12 heads, 32 k vocab) pretrained with whole‑word masking on CC‑100 and 2023‑Jan Wikipedia, using Unidic 2.1.2 word‑level tokenization plus WordPiece, in 2 M training steps.
- t5-base-japanese-v1.1sonoisa17kA Japanese T5‑v1.1 model pretrained on ≈100 GB of Wikipedia and OSCAR CC‑100 data (mixed 10:1 for SentencePiece with byte‑fallback), requiring fine‑tuning for downstream tasks, includes transfer‑learning sample code, notes potential bias in outputs, and is licensed CC‑BY‑SA 4.0.
- deberta-v3-base-japaneseku-nlp13kJapanese DeBERTa V3 base pre‑trained on 540 B tokens from LLM‑jp v1.0, trained with a modified DeBERTa V3 setup, uses a unigram byte‑fallback tokenizer (no morphological analyzer), and is fine‑tuned for JGLUE NLU tasks.
- t5-small-shortretrieva-jp3kA T5 v1.1 Japanese‑language model pre‑trained on mC4/ja and Wikipedia, featuring GEGLU activations, no dropout during pre‑training, separate embedding/classifier layers, and larger d_model with fewer heads, licensed CC‑BY‑SA 4.0 for commercial use.
- bert-large-japanese-char-v2tohoku-nlp2kA Japanese‑language BERT large model (24 layers, 1024 hidden size, 16 heads) pretrained on CC‑100 and Wikipedia using Unidic 2.1.2 word‑level tokenization followed by character splitting, with whole‑word masking for MLM, available via cl-tohoku/bert-japanese.
Resources in similar research fields
bert-japanesecl-tohoku551BERT models for Japanese language text.GitHubShared: Representation Learning・Language Models
Laboro-BERT-Japaneselaboroai73Laboro BERT Japanese: Japanese BERT Pre-Trained With Web-CorpusGitHubShared: Representation Learning・Language Models
language-pretrainingretarfi50BERT and ELECTRA models of PyTorch implementations for Japanese text.GitHubShared: Representation Learning・Language Models
bert-japanese-aozoraakirakubo41Japanese BERT trained on Aozora Bunko and Wikipedia, pre-tokenized by MeCab with UniDic & SudachiPyGitHubShared: Representation Learning・Language Models
albert-japanesealinear-corp33BERT with SentencePiece for Japanese text.GitHubShared: Representation Learning・Language Models
RoBERTa-japanesetanreinama23Japanese BERT Pretrained ModelGitHubShared: Representation Learning・Language Models