19

Conclusion

This survey systematically investigated and classified 898 GitHub repositories and 497 Hugging Face repositories in the field of Japanese NLP, using the awesome-japanese-nlp-resources repository as its foundation. The main findings are summarized below.

  1. Morphological analysis is still dominated by the MeCab family, but diversification is underway with the growth of SudachiPy (436k/week DL), the rise of Rust implementations, and the emergence of neural methods.

  2. In syntactic parsing, UD-compliant analysis is being standardized, centered on GiNZA (Stars: 878), making integration with multilingual NLP easier.

  3. In the OCR field, innovative Transformer-based methods such as manga-ocr (Stars: 2,800) and donut (Stars: 6,900) are attracting significant attention.

  4. In speech synthesis, VOICEVOX (combined Stars: 6,300+) has overwhelming popularity and has become an important component of Japan's content creation ecosystem.

  5. In LLM development, the infrastructure for developing domestic Japanese LLMs is being rapidly established, centered on the LLM-jp project and the Swallow project.

  6. Evaluation benchmarks are being developed, enabling multifaceted evaluation with JGLUE (Stars: 349), llm-jp-eval (Stars: 175), JMMLU (Stars: 45), and others.

  7. Fusion with Japanese pop culture — manga, games, and VTubers — has become a unique direction of development for Japanese NLP.

  8. Hugging Face has grown into the main hub for Japanese models, with 282 models and 215 datasets published, led by wav2vec2-large-xlsr-53-japanese (16M DL), japanese-gpt-neox-small (656k DL), ruri-v3-310m (634k DL), and plamo-embedding-1b (486k DL). Diverse demand continues across ASR, BERT models, and sentence embeddings, with the Ruri series remaining the standard choice for text embedding.

  9. Local LLM execution via GGUF quantization is becoming widespread, with many Japanese LLMs being provided in llama.cpp-compatible format.

Japanese NLP is at a transitional period in which traditional language processing technologies coexist with the latest deep learning technologies represented by LLMs. While foundational technologies such as morphological analysis will continue to be important, integration with LLMs and expansion into multimodal processing are expected to advance. The active activities of the open-source community continue to support the ongoing development of this field.