02

Morphological Analysis

Morphological analysis is the core technology of Japanese NLP — the process of segmenting text into words (morphemes) and attaching to each morpheme information such as part-of-speech, base form, and reading. Since Japanese text does not separate words with spaces, this processing is indispensable.

2.1 MeCab Family

MeCab is an open-source morphological analyzer based on Conditional Random Fields (CRF), developed by Taku Kudo and implemented in C++. Due to its speed and accuracy, it has long reigned as the de facto standard for Japanese NLP.

MeCab core (taku910/mecab): The original C++ implementation. Stars: 1,100. Last updated in February 2025, demonstrating its long-term stability.

Python bindings: Multiple bindings exist for using MeCab from Python, each with different characteristics.

Tool Weekly DL Total DL Stars Description
mecab-python3 174k 43M 581 The most standard Python binding
natto-py 2k 34M 95 ctypes-based binding; MeCab must be installed separately
fugashi 206k 20M 538 Cython implementation; fast and Pythonic

mecab-python3 has the highest weekly download count at 174k, with total downloads reaching 43M and Stars: 581. fugashi features Cython-based acceleration, with 206k/week downloads and 538 Stars. natto-py has 2k/week downloads and 34M total downloads.

mecab-python3
The most standard Python binding
581 174k/w
natto-py
ctypes-based binding; MeCab must be installed separately
95 2k/w
fugashi
Cython implementation; fast and Pythonic
538 206k/w
MeCab core
MeCab core (taku910/mecab): The original C++ implementation. Stars: 1,100. Last updated in February 2025, demonstrating its long-term stability.
1.1k

2.2 Sudachi Family

Sudachi is a Japanese morphological analyzer developed by Works Applications, designed with business use cases in mind. Its most distinctive feature is the ability to select the segmentation unit in three levels: A (short unit), B (medium unit), and C (long unit).

SudachiPy/sudachi.rs: Originally a Pure Python implementation (SudachiPy), but since version 0.6 it has migrated to a Rust implementation (sudachi.rs), achieving a significant speed improvement. Its weekly download count of 436k is the highest in the morphological analysis category, and total downloads of 70M demonstrate its wide adoption. Stars: 442.

SudachiDict (SudachiDict): Dictionary package for Sudachi. Three types are provided: small, core, and full. Stars: 318. Regular updates continue.

Sudachi (Java version) (WorksApplications/Sudachi): The original Java implementation. Stars: 1,000. Used for integration with Elasticsearch and Solr.

SudachiDict
SudachiDict (SudachiDict): Dictionary package for Sudachi. Three types are provided: small, core, and full. Stars: 318. Regular updates continue.
318
Sudachi (Java version)
Sudachi (Java version) (WorksApplications/Sudachi): The original Java implementation. Stars: 1,000. Used for integration with Elasticsearch and Solr.
1k

2.3 Juman++ Family

Juman++ is a morphological analyzer developed by the Kurohashi-Chu-Murawaki Laboratory at Kyoto University, featuring high-precision analysis using a Recurrent Neural Network Language Model (RNNLM).

Tool Stars Language Description
jumanpp 413 C++ Core. High-precision analysis using RNNLM
juman 12 C++ Old version
pyknp 93 Python Python module for Juman++/KNP (2k/week DL)
rhoknp 40 Python New binding for Juman++/KNP (28k/week DL)

rhoknp shows 28k downloads per week, far surpassing pyknp (2k/week). Active updates continue (last updated January 2026).

jumanpp
Core. High-precision analysis using RNNLM
413
juman
Old version
12
pyknp
Python module for Juman++/KNP (2k/week DL)
93
rhoknp
New binding for Juman++/KNP (28k/week DL)
40

2.4 Other Morphological Analysis Tools

Tool Weekly DL Total DL Stars Description
Janome 66k 14M 922 Pure Python implementation. Easy to install
nagisa 62k 11M 424 RNN-based. Includes a built-in pretrained model with POS tags
konoha 28k 7M 266 Unified wrapper for multiple tokenizers
Mykytea-python 4k 597k 36 Python wrapper for KyTea

Janome has 922 Stars, the highest among morphological analysis tools, and is valued for its ease of use as a Pure Python implementation. The fact that it can be used with just a pip install and no external dependencies makes it suitable for educational use and prototyping.

nagisa is an RNN-based tokenizer featuring POS estimation using BLSTM. It sees stable usage with 62k downloads per week.

Janome
Pure Python implementation. Easy to install
922 66k/w
nagisa
RNN-based. Includes a built-in pretrained model with POS tags
424 62k/w
konoha
Unified wrapper for multiple tokenizers
266 28k/w
Mykytea-python
Python wrapper for KyTea
36 4k/w

2.5 Comparison of Multilingual Implementations

Morphological analysis has been implemented in multiple programming languages, allowing selection based on use case.

Rust Implementations

Tool Total DL Stars Description
lindera 3.5M 708 MeCab-compatible. Strong search engine integration
vibrato 115k 430 Viterbi-based high-speed tokenizer
vaporetto 379k 302 Pointwise prediction-based
kanpyo 2.5k 108 Pure Rust morphological analyzer
goya 12k 84 Rust morphological analysis

Rust implementations have been rapidly developing in recent years, with lindera (Stars: 708) being the most popular. Integration with the Tantivy search engine (lindera-tantivy) is also provided. vibrato and vaporetto are high-performance implementations from the daac-tools group, each of which also provides a Python wrapper (python-vibrato, python-vaporetto).

lindera
MeCab-compatible. Strong search engine integration
708
vibrato
Viterbi-based high-speed tokenizer
430
vaporetto
Pointwise prediction-based
302
kanpyo
Pure Rust morphological analyzer
108
goya
Rust morphological analysis
84

Go Implementations

kagome (ikawaha/kagome): A self-contained morphological analyzer written in Pure Go, with Stars: 988 — extremely popular for a Go NLP tool. It can be used without external dependencies and is easy to embed in Go-based applications.

kagome
kagome (ikawaha/kagome): A self-contained morphological analyzer written in Pure Go, with Stars: 988 — extremely popular for a Go NLP tool. It can be used without external dependencies and is easy to embed in Go-based applications.
988

Java Implementations

kuromoji (atilika/kuromoji): A Java morphological analyzer also adopted by Apache Lucene/Solr, with Stars: 1,100 and high name recognition. It has strengths in search engine integration.

kuromoji
kuromoji (atilika/kuromoji): A Java morphological analyzer also adopted by Apache Lucene/Solr, with Stars: 1,100 and high name recognition. It has strengths in search engine integration.
1.1k

JavaScript Implementations

kuromoji.js (takuyaa/kuromoji.js): A JavaScript implementation with 319k weekly downloads and 11M total downloads, widely used in browser and Node.js environments. Stars: 1,000.

kuromoji.js
kuromoji.js (takuyaa/kuromoji.js): A JavaScript implementation with 319k weekly downloads and 11M total downloads, widely used in browser and Node.js environments. Stars: 1,000.
1k 319k/w

The following trends can be read from the usage patterns of morphological analysis tools:

  1. MeCab family still holds the largest share: The combined weekly downloads of MeCab-related bindings (mecab-python3, natto-py, fugashi) reach 424k, maintaining wide adoption.
  2. Growth of SudachiPy: Its 436k/week downloads now slightly exceed the combined MeCab tools, driven by the performance gains from the Rust migration and the flexibility of three-level segmentation.
  3. Rise of Rust implementations: Rust implementations such as lindera, vibrato, and vaporetto are being actively developed and increasingly adopted in use cases that prioritize performance and memory efficiency.
  4. Emergence of neural methods: Deep learning-based methods such as nagisa (BLSTM) and Mecari (GNN) are also appearing and being actively developed.