20
References
Repositories and Data Sources
- taishi-i, "awesome-japanese-nlp-resources," GitHub, 2026. https://github.com/taishi-i/awesome-japanese-nlp-resources
- taishi-i, "awesome-japanese-nlp-resources Japanese version," 2026. https://github.com/taishi-i/awesome-japanese-nlp-resources/blob/main/docs/README.ja.md
- taishi-i, "awesome-japanese-nlp-resources Hugging Face version," 2026. https://github.com/taishi-i/awesome-japanese-nlp-resources/blob/main/docs/huggingface.ja.md
Morphological Analysis
- Kudo, T., "MeCab: Yet Another Part-of-Speech and Morphological Analyzer," https://github.com/taku910/mecab
- Takaoka, K. et al., "Sudachi: a Japanese Tokenizer for Business," Proc. LREC, 2018. https://github.com/WorksApplications/Sudachi
- Tolmachev, A. et al., "Juman++ v2: A Practical and Modern Morphological Analyzer," Proc. ANLP, 2018. https://github.com/ku-nlp/jumanpp
- mocobeta, "Janome: Japanese Morphological Analysis Engine Written in Pure Python," https://github.com/mocobeta/janome
- Igarashi, T., "nagisa: A Japanese Tokenizer Based on Recurrent Neural Networks," https://github.com/taishi-i/nagisa
- polm, "fugashi: A Cython MeCab Wrapper," https://github.com/polm/fugashi
Syntactic Parsing
- Megagon Labs, "GiNZA: Japanese NLP Library using spaCy," https://github.com/megagonlabs/ginza
- Kudo, T., "CaboCha: Yet Another Japanese Dependency Structure Analyzer," https://github.com/taku910/cabocha
- Ku-NLP, "KNP: Japanese Parser," https://github.com/ku-nlp/knp
- Yoshikawa, M. et al., "depccg: A* CCG Parser," https://github.com/masashi-y/depccg
Pretrained Models
- Devlin, J. et al., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," NAACL, 2019.
- cl-tohoku, "bert-japanese," https://github.com/cl-tohoku/bert-japanese
- Yamada, I. et al., "LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention," EMNLP, 2020. https://github.com/studio-ousia/luke
- tanreinama, "gpt2-japanese," https://github.com/tanreinama/gpt2-japanese
- sonoisa, "t5-japanese," https://github.com/sonoisa/t5-japanese
Large Language Models and Evaluation
- LLM-jp, "llm-jp-eval," https://github.com/llm-jp/llm-jp-eval
- Yahoo Japan, "JGLUE: Japanese General Language Understanding Evaluation," https://github.com/yahoojapan/JGLUE
- Swallow LLM, "swallow-evaluation," https://github.com/swallow-llm/swallow-evaluation
- NLP Waseda, "JMMLU," https://github.com/nlp-waseda/JMMLU
OCR
- kha-white, "manga-ocr," https://github.com/kha-white/manga-ocr
- Kim, G. et al., "Donut: Document Understanding Transformer," ECCV, 2022. https://github.com/clovaai/donut
Speech Processing
- VOICEVOX, "voicevox_engine," https://github.com/VOICEVOX/voicevox_engine
- Reazon Research, "ReazonSpeech," https://github.com/reazon-research/reazonspeech
- sarulab-speech, "jtubespeech," https://github.com/sarulab-speech/jtubespeech
Dictionaries and IME
- neologd, "mecab-ipadic-neologd," https://github.com/neologd/mecab-ipadic-neologd
- Google, "Mozc," https://github.com/google/mozc
- polm, "unidic-py," https://github.com/polm/unidic-py
Sentence Embeddings and Retrieval
- answerdotai, "JaColBERTv2.5," https://huggingface.co/answerdotai/JaColBERTv2.5
- cl-nagoya, "Ruri: Japanese General Text Embeddings," https://huggingface.co/cl-nagoya/ruri-v3-310m
- pkshatech, "GLuCoSE-base-ja," https://huggingface.co/pkshatech/GLuCoSE-base-ja
- hotchpotch, "Japanese Reranker," https://huggingface.co/hotchpotch/japanese-reranker-cross-encoder-xsmall-v1
LLMs on Hugging Face
- llm-jp, "llm-jp-4-8b-instruct," https://huggingface.co/llm-jp/llm-jp-4-8b-instruct
- tokyotech-llm, "Llama-3-Swallow-8B-Instruct," https://huggingface.co/tokyotech-llm/Llama-3-Swallow-8B-Instruct-v0.1
- elyza, "Llama-3-ELYZA-JP-8B," https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B
- SakanaAI, "TinySwallow-1.5B," https://huggingface.co/SakanaAI/TinySwallow-1.5B
- LiquidAI, "LFM2.5-1.2B-JP," https://huggingface.co/LiquidAI/LFM2.5-1.2B-JP
Corpora
- Kurihara, K. et al., "JGLUE: Japanese General Language Understanding Evaluation," LREC, 2022.
- Pryzant, R. et al., "JESC: Japanese-English Subtitle Corpus," LREC, 2018. https://github.com/rpryzant/JESC
- ids-cv, "WRIME," https://github.com/ids-cv/wrime
- SkelterLabsInc, "JaQuAD," https://github.com/SkelterLabsInc/JaQuAD
Hugging Face Datasets
- KakologArchives, "KakologArchives," https://huggingface.co/datasets/KakologArchives/KakologArchives
- sbintuitions, "JMTEB," https://huggingface.co/datasets/sbintuitions/JMTEB
- llm-jp, "jhle," https://huggingface.co/datasets/llm-jp/jhle
- hotchpotch, "fineweb-2-edu-japanese," https://huggingface.co/datasets/hotchpotch/fineweb-2-edu-japanese
This survey is based on information as of November 17, 2026. Statistical information for each repository (download counts, Star counts, etc.) is based on the awesome-japanese-nlp-resources repository data updated on November 17, 2026.