A Japanese T5‑v1.1 model pretrained on ≈100 GB of Wikipedia and OSCAR CC‑100 data (mixed 10:1 for SentencePiece with byte‑fallback), requiring fine‑tuning for downstream tasks, includes transfer‑learning sample code, notes potential bias in outputs, and is licensed CC‑BY‑SA 4.0.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
17,000
Hugging Face likes
11
Listed
—
Listed since the page was created
Downloads (30-day window) over time
Show as a table
| Month | DL (30d) |
|---|---|
| Dec 2025 | 15,000 |
| Jan 2026 | 14,000 |
| Feb 2026 | 13,000 |
| Mar 2026 | 16,000 |
| Apr 2026 | 15,000 |
| May 2026 | 15,000 |
| Jun 2026 | 16,000 |
| Jul 2026 | 16,000 |
| Aug 2026 | 17,000 |
| Sep 2026 | 17,000 |
| Oct 2026 | 15,000 |
| Nov 2026 | 17,000 |
Downloads over the 30 days before the last snapshot of each month.
Popular in the same category
- bert-base-japanese-char-v3tohoku-nlp130kJapanese‑language BERT‑Base (12 layers, 768‑dim, 12 heads) pretrained with Unidic‑based word‑level plus character‑level tokenization and whole‑word masking on CC‑100 and 2023 Wikipedia, producing a 7,027‑token vocabulary.
- bert-large-japanese-v2tohoku-nlp66kJapanese‑BERT‑Large trained on CC‑100 and Wikipedia, using Unidic‑lite word‑level tokenization with WordPiece subwords and whole‑word masking (24 layers, 1024‑dim hidden, 16 heads, 32k vocab), with pretraining code on cl‑tohoku/bert‑japanese.
- bert-base-japanese-v3tohoku-nlp66kJapanese BERT‑base (12 layers, 768‑dim hidden, 12 heads, 32 k vocab) pretrained with whole‑word masking on CC‑100 and 2023‑Jan Wikipedia, using Unidic 2.1.2 word‑level tokenization plus WordPiece, in 2 M training steps.
- deberta-v3-base-japaneseku-nlp13kJapanese DeBERTa V3 base pre‑trained on 540 B tokens from LLM‑jp v1.0, trained with a modified DeBERTa V3 setup, uses a unigram byte‑fallback tokenizer (no morphological analyzer), and is fine‑tuned for JGLUE NLU tasks.
- t5-small-shortretrieva-jp3kA T5 v1.1 Japanese‑language model pre‑trained on mC4/ja and Wikipedia, featuring GEGLU activations, no dropout during pre‑training, separate embedding/classifier layers, and larger d_model with fewer heads, licensed CC‑BY‑SA 4.0 for commercial use.
- bert-large-japanese-char-v2tohoku-nlp2kA Japanese‑language BERT large model (24 layers, 1024 hidden size, 16 heads) pretrained on CC‑100 and Wikipedia using Unidic 2.1.2 word‑level tokenization followed by character splitting, with whole‑word masking for MLM, available via cl-tohoku/bert-japanese.
Resources in similar research fields
gpt2-japanesetanreinama322Japanese GPT2 Generation ModelGitHubShared: Text Generation・Language Models- modernbert-ja-310msbintuitions36kModernBERT‑Ja‑310M is a Japanese‑language BERT variant that blends local‑global attention with RoPE, trained on 4.09 T tokens of Japanese/English text, supports a 102 400‑word vocabulary, 8 192‑token sequences, and is optimized for Flash Attention 2.Shared: Language Models・Semantic Text Processing
- deberta-v2-base-japaneseku-nlp24kJapanese DeBERTa V2 base model pretrained on 171 GB of Japanese Wikipedia, CC‑100 and OSCAR data using Juman++ segmentation and SentencePiece tokenization, trained for three weeks on eight NVIDIA A100 GPUs and ready for fine‑tuning.Shared: Language Models・Semantic Text Processing
- modernbert-ja-130msbintuitions12kA 132‑million‑parameter Japanese ModernBERT model that blends local‑global and RoPE attention, trained on 4.39 T tokens (Japanese/English) with a 102‑k‑size vocab, 8,192‑token max length, and optimized for Flash Attention 2.Shared: Language Models・Semantic Text Processing
t5-japanesemegagonlabs39Codes to pre-train Japanese T5 modelsGitHubShared: Text Generation・Language Models
GPTSANtanreinama119General-purpose Swich transformer based Japanese language modeGitHubShared: Language Models・Semantic Text Processing