A Japanese RoBERTa base model pretrained on Wikipedia and CC‑100, fine‑tunable via Hugging Face, using BertJapaneseTokenizer with optional Juman++ tokenization and trained over 700k steps on eight A100 GPUs.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
2,000
Hugging Face likes
8
Listed
—
Listed since the page was created
Downloads (30-day window) over time
Show as a table
| Month | DL (30d) |
|---|---|
| Dec 2025 | 2,000 |
| Jan 2026 | 1,000 |
| Feb 2026 | — |
| Mar 2026 | — |
| Apr 2026 | 5,000 |
| May 2026 | — |
| Jun 2026 | — |
| Jul 2026 | — |
| Aug 2026 | — |
| Sep 2026 | 1,000 |
| Oct 2026 | 2,000 |
| Nov 2026 | 2,000 |
Downloads over the 30 days before the last snapshot of each month.
Popular in the same category
- bert-base-japanese-whole-word-maskingtohoku-nlp462kJapanese BERT‑base pretrained on 2019 Japanese Wikipedia with IPA‑dictionary and whole‑word masking, 12‑layer 768‑dim, 32,000‑vocab, 512‑token sequences, 1 M steps, available at cl‑tohoku/bert‑japanese under CC‑BY‑SA.
- deberta-v2-large-japanese-char-wwmku-nlp76kJapanese DeBERTa V2 large model trained on 171 GB of Japanese Wikipedia, CC‑100, and OSCAR with character‑level sentencepiece tokenization and whole‑word masking, ready for downstream fine‑tuning through Hugging Face Transformers.
- bert-base-japanese-char-v2tohoku-nlp55kA BERT‑base Japanese model (12 layers, 768‑dim hidden states, 12 heads) trained on 30 M Wikipedia sentences (~4 GB) with Unidic 2.1.2 word‑level tokenization followed by character‑level tokenization and whole‑word masking, using 512‑token sequences, 256 batches, and 1 M training steps.
- modernbert-ja-310msbintuitions36kModernBERT‑Ja‑310M is a Japanese‑language BERT variant that blends local‑global attention with RoPE, trained on 4.09 T tokens of Japanese/English text, supports a 102 400‑word vocabulary, 8 192‑token sequences, and is optimized for Flash Attention 2.
- deberta-v2-base-japanese-char-wwmku-nlp24kA Japanese DeBERTa‑V2 base model pre‑trained on 171 GB of Japanese Wikipedia, CC‑100, and OSCAR text with character‑level tokenization, whole‑word masking, trained for 20 days on 8 A100 GPUs, and ready for downstream fine‑tuning.
- deberta-v2-base-japaneseku-nlp24kJapanese DeBERTa V2 base model pretrained on 171 GB of Japanese Wikipedia, CC‑100 and OSCAR data using Juman++ segmentation and SentencePiece tokenization, trained for three weeks on eight NVIDIA A100 GPUs and ready for fine‑tuning.
Resources in similar research fields
- bert-base-japanese-char-v3tohoku-nlp130kJapanese‑language BERT‑Base (12 layers, 768‑dim, 12 heads) pretrained with Unidic‑based word‑level plus character‑level tokenization and whole‑word masking on CC‑100 and 2023 Wikipedia, producing a 7,027‑token vocabulary.Shared: Pretrained encoders (BERT etc.)
- bert-large-japanese-v2tohoku-nlp66kJapanese‑BERT‑Large trained on CC‑100 and Wikipedia, using Unidic‑lite word‑level tokenization with WordPiece subwords and whole‑word masking (24 layers, 1024‑dim hidden, 16 heads, 32k vocab), with pretraining code on cl‑tohoku/bert‑japanese.Shared: Pretrained encoders (BERT etc.)
- bert-base-japanese-v3tohoku-nlp66kJapanese BERT‑base (12 layers, 768‑dim hidden, 12 heads, 32 k vocab) pretrained with whole‑word masking on CC‑100 and 2023‑Jan Wikipedia, using Unidic 2.1.2 word‑level tokenization plus WordPiece, in 2 M training steps.Shared: Pretrained encoders (BERT etc.)
- t5-base-japanese-v1.1sonoisa17kA Japanese T5‑v1.1 model pretrained on ≈100 GB of Wikipedia and OSCAR CC‑100 data (mixed 10:1 for SentencePiece with byte‑fallback), requiring fine‑tuning for downstream tasks, includes transfer‑learning sample code, notes potential bias in outputs, and is licensed CC‑BY‑SA 4.0.Shared: Pretrained encoders (BERT etc.)
- line-distilbert-base-japaneseline-corporation14kLINE DistilBERT Japanese is a 66‑million‑parameter DistilBERT model pre‑trained on 131 GB of Japanese web text using an in‑house BERT‑base teacher, evaluated on JGLUE, tokenized with MeCab Unidic and SentencePiece, and released under the Apache 2.0 license.Shared: Pretrained encoders (BERT etc.)
- deberta-v2-tiny-japaneseku-nlp14kJapanese DeBERTa V2 tiny, pretrained on ~171 GB of Japanese Wikipedia, CC‑100, and OSCAR corpora, requires Juman++ word segmentation, was trained in 33 h on 8 NVIDIA A100 GPUs, and can be fine‑tuned for downstream tasks.Shared: Pretrained encoders (BERT etc.)