A T5 v1.1 Japanese‑language model pre‑trained on mC4/ja and Wikipedia, featuring GEGLU activations, no dropout during pre‑training, separate embedding/classifier layers, and larger d_model with fewer heads, licensed CC‑BY‑SA 4.0 for commercial use.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
3,000
Hugging Face likes
3
Listed
May 13, 2024
Downloads (30-day window) over time
Show as a table
| Month | DL (30d) |
|---|---|
| Dec 2025 | 2,000 |
| Jan 2026 | — |
| Feb 2026 | — |
| Mar 2026 | 1,000 |
| Apr 2026 | 3,000 |
| May 2026 | 3,000 |
| Jun 2026 | — |
| Jul 2026 | 6,000 |
| Aug 2026 | 2,000 |
| Sep 2026 | 10,000 |
| Oct 2026 | 4,000 |
| Nov 2026 | 3,000 |
Downloads over the 30 days before the last snapshot of each month.
Popular in the same category
- bert-base-japanese-char-v3tohoku-nlp130kJapanese‑language BERT‑Base (12 layers, 768‑dim, 12 heads) pretrained with Unidic‑based word‑level plus character‑level tokenization and whole‑word masking on CC‑100 and 2023 Wikipedia, producing a 7,027‑token vocabulary.
- bert-large-japanese-v2tohoku-nlp66kJapanese‑BERT‑Large trained on CC‑100 and Wikipedia, using Unidic‑lite word‑level tokenization with WordPiece subwords and whole‑word masking (24 layers, 1024‑dim hidden, 16 heads, 32k vocab), with pretraining code on cl‑tohoku/bert‑japanese.
- bert-base-japanese-v3tohoku-nlp66kJapanese BERT‑base (12 layers, 768‑dim hidden, 12 heads, 32 k vocab) pretrained with whole‑word masking on CC‑100 and 2023‑Jan Wikipedia, using Unidic 2.1.2 word‑level tokenization plus WordPiece, in 2 M training steps.
- t5-base-japanese-v1.1sonoisa17kA Japanese T5‑v1.1 model pretrained on ≈100 GB of Wikipedia and OSCAR CC‑100 data (mixed 10:1 for SentencePiece with byte‑fallback), requiring fine‑tuning for downstream tasks, includes transfer‑learning sample code, notes potential bias in outputs, and is licensed CC‑BY‑SA 4.0.
- deberta-v3-base-japaneseku-nlp13kJapanese DeBERTa V3 base pre‑trained on 540 B tokens from LLM‑jp v1.0, trained with a modified DeBERTa V3 setup, uses a unigram byte‑fallback tokenizer (no morphological analyzer), and is fine‑tuned for JGLUE NLU tasks.
- japanese-splade-v2hotchpotch4kHigh‑performance Japanese SPLADE v2 enables sparse‑vector conversion and inference through a WebUI demo, trains with YAST, offers YASEM embedding, and reports JMTEB benchmark results.
Resources in similar research fields
- ruri-v3-310mcl-nagoya634kRuri v3 is a state‑of‑the‑art Japanese text‑embedding model built on ModernBERT‑Ja, supporting up to 8,192‑token inputs, a 100K‑token vocabulary, FlashAttention‑accelerated inference, and multiple size variants for fast sentence‑transformer usage.Shared: Embeddings, search & RAG・Pretrained encoders (BERT etc.)
- ruri-v3-reranker-310mcl-nagoya137kRuri‑v3 Reranker is a robust Japanese text reranker built on ModernBERT‑Ja, supporting up to 8,192‑token sequences, a 100k‑token vocabulary, FlashAttention and a SentencePiece tokenizer, and it can be used via sentence‑transformers.Shared: Embeddings, search & RAG・Pretrained encoders (BERT etc.)
- ruri-v3-30mcl-nagoya126kRuri v3 is a state‑of‑the‑art Japanese text embedding model built on ModernBERT‑Ja, supporting up to 8,192 tokens, a 100 k‑token vocabulary, FlashAttention acceleration, and multiple sizes from 37 M to 315 M parameters.Shared: Embeddings, search & RAG・Pretrained encoders (BERT etc.)
- ruri-v3-130mcl-nagoya93kRuri v3 is a state‑of‑the‑art Japanese text‑embedding model built on ModernBERT‑Ja, supporting up to 8192‑token sequences, a 100K‑token vocabulary, FlashAttention, and released in sizes from 30 M to 310 M parameters for use with sentence‑transformers.Shared: Embeddings, search & RAG・Pretrained encoders (BERT etc.)
- sentence-bert-base-ja-mean-tokens-v2sonoisa53kA Japanese Sentence‑BERT v2, fine‑tuned on cl‑tohoku/bert‑base‑japanese‑whole‑word‑masking with MultipleNegativesRankingLoss, boosts accuracy by ~1.5–2 % over v1 and is released as sonoisa/sentence‑bert‑base‑ja‑mean‑tokens‑v2.Shared: Embeddings, search & RAG・Pretrained encoders (BERT etc.)
- transformers-ud-japanese-electra-base-ginza-510megagonlabs22kja_ginza_electra is a spaCy v3 Python package offering a Japanese ELECTRA model fine‑tuned on mC4 and UD_Japanese_BCCWJ r2.8 (based on megagonlabs/transformers‑ud‑japanese‑electra‑base‑discrimininator) with custom bunsetu‑phrase detection, distributed under the MIT license.Shared: Embeddings, search & RAG・Pretrained encoders (BERT etc.)