09

Pretrained Models

9.1 Word2Vec and Static Embeddings

Tool Stars Description
chiVe 179 Word embeddings using Sudachi + NWJC
japanese-words-to-vectors 87 Word2Vec using Gensim + MeCab
embedrank 48 Python implementation of EmbedRank

chiVe is a Japanese word embedding provided by Works Applications, using Sudachi tokenization and the NINJAL Web Japanese Corpus (NWJC) as training data. Stars: 179.

chiVe
Word embeddings using Sudachi + NWJC
179
japanese-words-to-vectors
Word2Vec using Gensim + MeCab
87
embedrank
Python implementation of EmbedRank
48

9.2 BERT-family Models

Model Stars Developer Description
bert-japanese (cl-tohoku) 551 Tohoku University Trained on Japanese Wikipedia. Most widely used
bert-japanese (yoheikikuta) 497 Individual Uses SentencePiece
LUKE 727 Studio Ousia Integrates entity embeddings
Laboro-BERT-Japanese 73 Laboro.AI Trained on web corpus
DistilBERT-base-jp 161 Bandai Namco Research Lightweight BERT
LINE-DistilBERT-Japanese 47 LINE Trained on 131GB of web text
shiba 89 Octanove Labs CANINE (character-level Transformer)
AcademicRoBERTa 10 Ehime University Trained on academic paper abstracts

Tohoku University BERT (bert-japanese) is the most widely used Japanese BERT model with Stars: 551. It supports Whole Word Masking and provides two tokenization variants: MeCab+WordPiece and Sudachi+WordPiece. On Hugging Face, tohoku-nlp/bert-base-japanese-whole-word-masking shows the highest usage at 462k/week downloads (76 Likes), followed by ku-nlp/deberta-v2-large-japanese-char-wwm at 76k/week (9 Likes) and tohoku-nlp/bert-base-japanese at 9k/week (42 Likes).

LUKE is a language understanding model developed by Studio Ousia that integrates knowledge base embeddings, with Stars: 727 — the highest popularity among BERT-family models. It shows particularly superior performance on entity recognition tasks.

bert-japanese (cl-tohoku)
Trained on Japanese Wikipedia. Most widely used
551
bert-japanese (yoheikikuta)
Uses SentencePiece
497
LUKE
Integrates entity embeddings
727
Laboro-BERT-Japanese
Trained on web corpus
73
DistilBERT-base-jp
Lightweight BERT
161
LINE-DistilBERT-Japanese
Trained on 131GB of web text
47
shiba
CANINE (character-level Transformer)
89
AcademicRoBERTa
Trained on academic paper abstracts
10

Japanese BERT-family Models on Hugging Face

On Hugging Face, diverse models are published including various variants of Tohoku University BERT, DeBERTa-family, RoBERTa-family, and ModernBERT-family.

Model HF DL Likes Description
bert-base-japanese (tohoku-nlp) 9k 42 Standard Japanese BERT
bert-base-japanese-whole-word-masking (tohoku-nlp) 462k 76 IPA dictionary + whole word masking
bert-base-japanese-char (tohoku-nlp) 11k 8 Character-level tokenization
bert-large-japanese-v2 (tohoku-nlp) 66k 14 Large-scale training on CC-100+Wikipedia
deberta-v2-large-japanese-char-wwm (ku-nlp) 76k 9 Trained on 171GB Japanese data
modernbert-ja-130m (sbintuitions) 12k 51 RoPE + local/global attention
modernbert-ja-310m (sbintuitions) 36k 32 Supports up to 8192 tokens
line-distilbert-base-japanese (LINE) 14k 50 Lightweight 66M-parameter version
jmedroberta-base-sentencepiece-vocab50000 (alabnii) 1k 1 Pretrained on medical paper abstracts
deberta-v2-base-japanese (ku-nlp) 24k 30 Base model trained on 171GB of Japanese text

Particularly noteworthy is the emergence of the ModernBERT-Ja series (sbintuitions). It uses the latest architecture combining RoPE and local/global attention mechanisms, supporting sequence lengths of up to 8,192 tokens. Multiple sizes from 30M to 310M are provided, optimized for Flash Attention 2.

Also, as a medical-domain BERT, jmedroberta-base-sentencepiece (3k/week DL) has been pretrained on approximately 10M Japanese medical abstracts, showing progress in deployment to specialized domains.

bert-base-japanese (tohoku-nlp)
Standard Japanese BERT
9k 42
bert-base-japanese-whole-word-masking (tohoku-nlp)
IPA dictionary + whole word masking
462k 76
bert-base-japanese-char (tohoku-nlp)
Character-level tokenization
11k 8
bert-large-japanese-v2 (tohoku-nlp)
Large-scale training on CC-100+Wikipedia
66k 14
deberta-v2-large-japanese-char-wwm (ku-nlp)
Trained on 171GB Japanese data
76k 9
modernbert-ja-130m (sbintuitions)
RoPE + local/global attention
12k 51
modernbert-ja-310m (sbintuitions)
Supports up to 8192 tokens
36k 32
line-distilbert-base-japanese (LINE)
Lightweight 66M-parameter version
14k 50
jmedroberta-base-sentencepiece-vocab50000 (alabnii)
Pretrained on medical paper abstracts
1k 1
deberta-v2-base-japanese (ku-nlp)
Base model trained on 171GB of Japanese text
24k 30

9.3 GPT-family Models

Model Stars Description
gpt2-japanese 322 Japanese GPT-2 generative model
GPTSAN 119 General-purpose Switch Transformer-based
gpt-ja 3 GPT-2 Japanese for HuggingFace

gpt2-japanese (tanreinama) has Stars: 322, the highest name recognition among Japanese GPT-2 models, and is widely used for text generation tasks.

GPTSAN is a Japanese-compatible model based on the Switch Transformer, supporting mixed Japanese and English text.

gpt2-japanese
Japanese GPT-2 generative model
322
GPTSAN
General-purpose Switch Transformer-based
119
gpt-ja
GPT-2 Japanese for HuggingFace
3

Japanese GPT-family Models on Hugging Face

On Hugging Face, multiple organizations including rinna, CyberAgent, and ABEJA have published GPT-family models.

Model HF DL Likes Developer Description
japanese-gpt-neox-small (rinna) 656k 15 rinna Trained on CC-100/C4/Wikipedia
japanese-gpt2-medium (rinna) 6k 85 rinna 24 layers, 1024 units
japanese-gpt2-small (rinna) 4k 30 rinna 12 layers, 768 units
gpt-neox-japanese-2.7b (abeja) 3k 59 ABEJA Trained on CC-100/OSCAR
open-calm-small (cyberagent) 1k 21 CyberAgent Multiple sizes from 160M to 6.8B

OpenCALM (CyberAgent): A suite of decoder-only Japanese Transformer models, providing multiple sizes from 160M to 6.8B parameters. It is notably published under the CC BY-SA 4.0 license.

japanese-gpt-neox-small (rinna)
Trained on CC-100/C4/Wikipedia
656k 15
japanese-gpt2-medium (rinna)
24 layers, 1024 units
6k 85
japanese-gpt2-small (rinna)
12 layers, 768 units
4k 30
gpt-neox-japanese-2.7b (abeja)
Trained on CC-100/OSCAR
3k 59
open-calm-small (cyberagent)
Multiple sizes from 160M to 6.8B
1k 21
OpenCALM
OpenCALM (CyberAgent): A suite of decoder-only Japanese Transformer models, providing multiple sizes from 160M to 6.8B parameters. It is notably published under the CC BY-SA 4.0 license.

9.4 T5-family Models

t5-japanese (sonoisa/t5-japanese): A Japanese T5 model. Stars: 118. Applicable to diverse tasks including text generation, summarization, and question answering.

t5-japanese (megagonlabs/t5-japanese): Japanese T5 model pretraining code by Megagon Labs. Stars: 40.

t5-japanese
t5-japanese (sonoisa/t5-japanese): A Japanese T5 model. Stars: 118. Applicable to diverse tasks including text generation, summarization, and question answering.
118
Model Stars Description
Japanese-Alpaca-LoRA 141 Japanese LoRA adapter for LLaMA
japanese-llama-experiment 54 Japanese LLaMA experiments
easylightchatassistant 48 Lightweight local Japanese LLM

Japanese-Alpaca-LoRA is a LoRA adapter fine-tuned on LLaMA using a Japanese translation of the Stanford Alpaca dataset, and attracted significant attention as an early experiment in Japanese LLMs.

Japanese-Alpaca-LoRA
Japanese LoRA adapter for LLaMA
141
japanese-llama-experiment
Japanese LLaMA experiments
54
easylightchatassistant
Lightweight local Japanese LLM
48

9.6 Evaluation Frameworks

Tool Stars Description
JGLUE 349 Japanese version of GLUE. Provided by Yahoo! JAPAN
llm-jp-eval 175 Automated evaluation across multiple datasets
swallow-evaluation 26 Swallow project evaluation
llm-leaderboard 95 LLM evaluation by W&B
jmteb 96 Japanese text embedding benchmark
japanese-llm-ranking 50 Camel leaderboard
JMMLU 45 Japanese Massive Multitask Language Understanding

JGLUE (yahoojapan/JGLUE) is the Japanese version of the GLUE benchmark provided by Yahoo! JAPAN, and is the most widely used among evaluation frameworks at Stars: 349. It includes tasks covering text classification, sentence pair classification, and question answering.

llm-jp-eval is an automated evaluation tool for Japanese LLMs by the LLM-jp project, performing evaluation across multiple datasets. Stars: 175, with active updates continuing (as of February 2026).


JGLUE
Japanese version of GLUE. Provided by Yahoo! JAPAN
349
llm-jp-eval
Automated evaluation across multiple datasets
175
swallow-evaluation
Swallow project evaluation
26
llm-leaderboard
LLM evaluation by W&B
95
jmteb
Japanese text embedding benchmark
96
japanese-llm-ranking
Camel leaderboard
50
JMMLU
Japanese Massive Multitask Language Understanding
45