Morphological Analysis
Morphological analysis is the core technology of Japanese NLP — the process of segmenting text into words (morphemes) and attaching to each morpheme information such as part-of-speech, base form, and reading. Since Japanese text does not separate words with spaces, this processing is indispensable.
2.1 MeCab Family
MeCab is an open-source morphological analyzer based on Conditional Random Fields (CRF), developed by Taku Kudo and implemented in C++. Due to its speed and accuracy, it has long reigned as the de facto standard for Japanese NLP.
MeCab core (taku910/mecab): The original C++ implementation. Stars: 1,100. Last updated in February 2025, demonstrating its long-term stability.
Python bindings: Multiple bindings exist for using MeCab from Python, each with different characteristics.
| Tool | Weekly DL | Total DL | Stars | Description |
|---|---|---|---|---|
| mecab-python3 | 174k | 43M | 581 | The most standard Python binding |
| natto-py | 2k | 34M | 95 | ctypes-based binding; MeCab must be installed separately |
| fugashi | 206k | 20M | 538 | Cython implementation; fast and Pythonic |
mecab-python3 has the highest weekly download count at 174k, with total downloads reaching 43M and Stars: 581. fugashi features Cython-based acceleration, with 206k/week downloads and 538 Stars. natto-py has 2k/week downloads and 34M total downloads.
2.2 Sudachi Family
Sudachi is a Japanese morphological analyzer developed by Works Applications, designed with business use cases in mind. Its most distinctive feature is the ability to select the segmentation unit in three levels: A (short unit), B (medium unit), and C (long unit).
SudachiPy/sudachi.rs: Originally a Pure Python implementation (SudachiPy), but since version 0.6 it has migrated to a Rust implementation (sudachi.rs), achieving a significant speed improvement. Its weekly download count of 436k is the highest in the morphological analysis category, and total downloads of 70M demonstrate its wide adoption. Stars: 442.
SudachiDict (SudachiDict): Dictionary package for Sudachi. Three types are provided: small, core, and full. Stars: 318. Regular updates continue.
Sudachi (Java version) (WorksApplications/Sudachi): The original Java implementation. Stars: 1,000. Used for integration with Elasticsearch and Solr.
2.3 Juman++ Family
Juman++ is a morphological analyzer developed by the Kurohashi-Chu-Murawaki Laboratory at Kyoto University, featuring high-precision analysis using a Recurrent Neural Network Language Model (RNNLM).
| Tool | Stars | Language | Description |
|---|---|---|---|
| jumanpp | 413 | C++ | Core. High-precision analysis using RNNLM |
| juman | 12 | C++ | Old version |
| pyknp | 93 | Python | Python module for Juman++/KNP (2k/week DL) |
| rhoknp | 40 | Python | New binding for Juman++/KNP (28k/week DL) |
rhoknp shows 28k downloads per week, far surpassing pyknp (2k/week). Active updates continue (last updated January 2026).
2.4 Other Morphological Analysis Tools
| Tool | Weekly DL | Total DL | Stars | Description |
|---|---|---|---|---|
| Janome | 66k | 14M | 922 | Pure Python implementation. Easy to install |
| nagisa | 62k | 11M | 424 | RNN-based. Includes a built-in pretrained model with POS tags |
| konoha | 28k | 7M | 266 | Unified wrapper for multiple tokenizers |
| Mykytea-python | 4k | 597k | 36 | Python wrapper for KyTea |
Janome has 922 Stars, the highest among morphological analysis tools, and is valued for its ease of use as a Pure Python implementation. The fact that it can be used with just a pip install and no external dependencies makes it suitable for educational use and prototyping.
nagisa is an RNN-based tokenizer featuring POS estimation using BLSTM. It sees stable usage with 62k downloads per week.
2.5 Comparison of Multilingual Implementations
Morphological analysis has been implemented in multiple programming languages, allowing selection based on use case.
Rust Implementations
| Tool | Total DL | Stars | Description |
|---|---|---|---|
| lindera | 3.5M | 708 | MeCab-compatible. Strong search engine integration |
| vibrato | 115k | 430 | Viterbi-based high-speed tokenizer |
| vaporetto | 379k | 302 | Pointwise prediction-based |
| kanpyo | 2.5k | 108 | Pure Rust morphological analyzer |
| goya | 12k | 84 | Rust morphological analysis |
Rust implementations have been rapidly developing in recent years, with lindera (Stars: 708) being the most popular. Integration with the Tantivy search engine (lindera-tantivy) is also provided. vibrato and vaporetto are high-performance implementations from the daac-tools group, each of which also provides a Python wrapper (python-vibrato, python-vaporetto).
Go Implementations
kagome (ikawaha/kagome): A self-contained morphological analyzer written in Pure Go, with Stars: 988 — extremely popular for a Go NLP tool. It can be used without external dependencies and is easy to embed in Go-based applications.
Java Implementations
kuromoji (atilika/kuromoji): A Java morphological analyzer also adopted by Apache Lucene/Solr, with Stars: 1,100 and high name recognition. It has strengths in search engine integration.
JavaScript Implementations
kuromoji.js (takuyaa/kuromoji.js): A JavaScript implementation with 319k weekly downloads and 11M total downloads, widely used in browser and Node.js environments. Stars: 1,000.
2.6 Overall Trends in Morphological Analysis Tools
The following trends can be read from the usage patterns of morphological analysis tools:
- MeCab family still holds the largest share: The combined weekly downloads of MeCab-related bindings (mecab-python3, natto-py, fugashi) reach 424k, maintaining wide adoption.
- Growth of SudachiPy: Its 436k/week downloads now slightly exceed the combined MeCab tools, driven by the performance gains from the Rust migration and the flexibility of three-level segmentation.
- Rise of Rust implementations: Rust implementations such as lindera, vibrato, and vaporetto are being actively developed and increasingly adopted in use cases that prioritize performance and memory efficiency.
- Emergence of neural methods: Deep learning-based methods such as nagisa (BLSTM) and Mecari (GNN) are also appearing and being actively developed.