
Pretrained models, codes and guidances to pretrain official ALBERT(https://github.com/google-research/albert) on Japanese Wikipedia Resources
Description in other languages
Statistics
Statistics as of Nov 17, 2026
GitHub stars
13
±0 (Oct 13〜)
Last commit
Sep 2023
Created on GitHub
Jun 3, 2023
Listed
Jun 11, 2023
Stars over time
Show as a table
| Month | Stars |
|---|---|
| Dec 2025 | 13 |
| Jan 2026 | 13 |
| Feb 2026 | 13 |
| Mar 2026 | 13 |
| Apr 2026 | 13 |
| May 2026 | 13 |
| Jun 2026 | 13 |
| Jul 2026 | 13 |
| Aug 2026 | 13 |
| Sep 2026 | 13 |
| Oct 2026 | 13 |
| Nov 2026 | 13 |
Last snapshot of each month. Star counts of 1,000 or more are rounded to the nearest 100.
Popular in the same category
gpt2-japanesetanreinama322Japanese GPT2 Generation Model
t5-japanesemegagonlabs39Codes to pre-train Japanese T5 models
text2text-japanesetanreinama34gpt-2 based text2text conversion model
gpt-jacolorfulscoop3GPT-2 Japanese model for HuggingFace's transformers
lukestudio-ousia727LUKE -- Language Understanding with Knowledge-based Embeddings
bert-japanesecl-tohoku551BERT models for Japanese language text.
Resources in similar research fields
jmlm_scoringminhpqn5Masked Language Model-based Scoring for Japanese and VietnameseShared: Text Generation・Language Models
t5_japanese_dialogue_generationJinyamyzk3Conversation generation using T5.Shared: Text Generation・Language Models- deberta-v3-base-japaneseku-nlp13kJapanese DeBERTa V3 base pre‑trained on 540 B tokens from LLM‑jp v1.0, trained with a modified DeBERTa V3 setup, uses a unigram byte‑fallback tokenizer (no morphological analyzer), and is fine‑tuned for JGLUE NLU tasks.HF modelShared: Language Models・Semantic Text Processing
- bert-large-japanese-v2tohoku-nlp66kJapanese‑BERT‑Large trained on CC‑100 and Wikipedia, using Unidic‑lite word‑level tokenization with WordPiece subwords and whole‑word masking (24 layers, 1024‑dim hidden, 16 heads, 32k vocab), with pretraining code on cl‑tohoku/bert‑japanese.HF modelShared: Representation Learning・Language Models
- bert-base-japanese-v3tohoku-nlp66kJapanese BERT‑base (12 layers, 768‑dim hidden, 12 heads, 32 k vocab) pretrained with whole‑word masking on CC‑100 and 2023‑Jan Wikipedia, using Unidic 2.1.2 word‑level tokenization plus WordPiece, in 2 M training steps.HF modelShared: Representation Learning・Language Models
- t5-base-japanese-v1.1sonoisa17kA Japanese T5‑v1.1 model pretrained on ≈100 GB of Wikipedia and OSCAR CC‑100 data (mixed 10:1 for SentencePiece with byte‑fallback), requiring fine‑tuning for downstream tasks, includes transfer‑learning sample code, notes potential bias in outputs, and is licensed CC‑BY‑SA 4.0.HF modelShared: Text Generation・Language Models