Japanese DeBERTa V3 base pre‑trained on 540 B tokens from LLM‑jp v1.0, trained with a modified DeBERTa V3 setup, uses a unigram byte‑fallback tokenizer (no morphological analyzer), and is fine‑tuned for JGLUE NLU tasks.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
13,000
Hugging Face likes
21
Listed
May 13, 2024
Downloads (30-day window) over time
Show as a table
| Month | DL (30d) |
|---|---|
| Dec 2025 | 16,000 |
| Jan 2026 | 13,000 |
| Feb 2026 | 6,000 |
| Mar 2026 | 3,000 |
| Apr 2026 | 21,000 |
| May 2026 | 3,000 |
| Jun 2026 | 5,000 |
| Jul 2026 | 10,000 |
| Aug 2026 | 4,000 |
| Sep 2026 | 5,000 |
| Oct 2026 | 4,000 |
| Nov 2026 | 13,000 |
Downloads over the 30 days before the last snapshot of each month.
Popular in the same category
- bert-base-japanese-char-v3tohoku-nlp130kJapanese‑language BERT‑Base (12 layers, 768‑dim, 12 heads) pretrained with Unidic‑based word‑level plus character‑level tokenization and whole‑word masking on CC‑100 and 2023 Wikipedia, producing a 7,027‑token vocabulary.
- bert-large-japanese-v2tohoku-nlp66kJapanese‑BERT‑Large trained on CC‑100 and Wikipedia, using Unidic‑lite word‑level tokenization with WordPiece subwords and whole‑word masking (24 layers, 1024‑dim hidden, 16 heads, 32k vocab), with pretraining code on cl‑tohoku/bert‑japanese.
- bert-base-japanese-v3tohoku-nlp66kJapanese BERT‑base (12 layers, 768‑dim hidden, 12 heads, 32 k vocab) pretrained with whole‑word masking on CC‑100 and 2023‑Jan Wikipedia, using Unidic 2.1.2 word‑level tokenization plus WordPiece, in 2 M training steps.
- t5-base-japanese-v1.1sonoisa17kA Japanese T5‑v1.1 model pretrained on ≈100 GB of Wikipedia and OSCAR CC‑100 data (mixed 10:1 for SentencePiece with byte‑fallback), requiring fine‑tuning for downstream tasks, includes transfer‑learning sample code, notes potential bias in outputs, and is licensed CC‑BY‑SA 4.0.
- Llama-3.1-Swallow-8B-Instruct-v0.5-ggufmmnga7kGGUF conversion of Llama‑3.1‑Swallow‑8B‑Instruct‑v0.5 by tokyotech‑llm, incorporating TFMC/imatrix‑dataset‑for‑japanese‑LLM, with Build/Run instructions for llama.cpp.
- sarashina2.2-3b-instruct-v0.1-ggufmmnga4kA GGUF‑formatted conversion of sbintuitions’ sarashina2.2‑3b‑instruct‑v0.1 model built with TFMC/imatrix‑dataset‑for‑japanese‑llm, ready for CUDA‑enabled inference via llama.cpp.
Resources in similar research fields
- japanese-gpt-neox-smallrinna656kA 12‑layer, 768‑hidden Japanese GPT‑NeoX model trained on CC‑100, C4, and Wikipedia, compatible with Huggingface, with an optional toy prefix‑tuning weight that forces each sentence to end with a smiling‑face emoji.Shared: Language Models・Semantic Text Processing
- llm-jp-3-150mllm-jp75kLLM‑jp‑3‑150m—a 150‑M‑parameter Japanese language model from the National Institute of Informatics’ LLM R&D Center—is distributed in Hugging Face Transformers format, requiring torch ≥ 2.3.0, transformers ≥ 4.40.1, accelerate ≥ 0.29.3, flash‑attn ≥ 2.5.8, and was pretrained on Japanese Wikipedia, Common Crawl, WARP/PDF, WARP/HTML, and Kaken data using a unigram byte‑fallback tokenizer.Shared: Language Models・Semantic Text Processing
- modernbert-ja-310msbintuitions36kModernBERT‑Ja‑310M is a Japanese‑language BERT variant that blends local‑global attention with RoPE, trained on 4.09 T tokens of Japanese/English text, supports a 102 400‑word vocabulary, 8 192‑token sequences, and is optimized for Flash Attention 2.Shared: Language Models・Semantic Text Processing
- deberta-v2-base-japaneseku-nlp24kJapanese DeBERTa V2 base model pretrained on 171 GB of Japanese Wikipedia, CC‑100 and OSCAR data using Juman++ segmentation and SentencePiece tokenization, trained for three weeks on eight NVIDIA A100 GPUs and ready for fine‑tuning.Shared: Language Models・Semantic Text Processing
- modernbert-ja-130msbintuitions12kA 132‑million‑parameter Japanese ModernBERT model that blends local‑global and RoPE attention, trained on 4.39 T tokens (Japanese/English) with a 102‑k‑size vocab, 8,192‑token max length, and optimized for Flash Attention 2.Shared: Language Models・Semantic Text Processing
GPTSANtanreinama119General-purpose Swich transformer based Japanese language modeGitHubShared: Language Models・Semantic Text Processing