
A comparison tool of Japanese tokenizers
Description in other languages
Statistics
Statistics as of Nov 17, 2026
GitHub stars
122
±0 (Oct 13〜)
Package downloads per week
28
Total package downloads
29,000
Last commit
Nov 2025
Created on GitHub
Aug 13, 2020
Listed
—
Listed since the page was created
Stars over time
Show as a table
| Month | Stars |
|---|---|
| Dec 2025 | 120 |
| Jan 2026 | 120 |
| Feb 2026 | 121 |
| Mar 2026 | 121 |
| Apr 2026 | 121 |
| May 2026 | 123 |
| Jun 2026 | 122 |
| Jul 2026 | 122 |
| Aug 2026 | 122 |
| Sep 2026 | 122 |
| Oct 2026 | 122 |
| Nov 2026 | 122 |
Last snapshot of each month. Star counts of 1,000 or more are rounded to the nearest 100.
Use on PyPI
PyPI packages: toiro · Monthly figures to Oct 2026 (ClickPy)
Downloads, last 12 months
4,590
Share from Japan (last 12 months)
2.1%
PyPI packages that depend on it
0
0 as a required dependency
Monthly downloads on PyPI
Show as a table
| Month | downloads (month) |
|---|---|
| Jan 2019 | — |
| Feb 2019 | — |
| Mar 2019 | — |
| Apr 2019 | — |
| May 2019 | — |
| Jun 2019 | — |
| Jul 2019 | — |
| Aug 2019 | — |
| Sep 2019 | — |
| Oct 2019 | — |
| Nov 2019 | — |
| Dec 2019 | — |
| Jan 2020 | — |
| Feb 2020 | — |
| Mar 2020 | — |
| Apr 2020 | — |
| May 2020 | — |
| Jun 2020 | — |
| Jul 2020 | — |
| Aug 2020 | 1,726 |
| Sep 2020 | 1,152 |
| Oct 2020 | 630 |
| Nov 2020 | 906 |
| Dec 2020 | 504 |
| Jan 2021 | 547 |
| Feb 2021 | 531 |
| Mar 2021 | 583 |
| Apr 2021 | 364 |
| May 2021 | 450 |
| Jun 2021 | 410 |
| Jul 2021 | 458 |
| Aug 2021 | 298 |
| Sep 2021 | 343 |
| Oct 2021 | 383 |
| Nov 2021 | 297 |
| Dec 2021 | 360 |
| Jan 2022 | 406 |
| Feb 2022 | 391 |
| Mar 2022 | 660 |
| Apr 2022 | 429 |
| May 2022 | 281 |
| Jun 2022 | 353 |
| Jul 2022 | 249 |
| Aug 2022 | 346 |
| Sep 2022 | 358 |
| Oct 2022 | 364 |
| Nov 2022 | 384 |
| Dec 2022 | 203 |
| Jan 2023 | 148 |
| Feb 2023 | 349 |
| Mar 2023 | 330 |
| Apr 2023 | 266 |
| May 2023 | 327 |
| Jun 2023 | 186 |
| Jul 2023 | 451 |
| Aug 2023 | 477 |
| Sep 2023 | 241 |
| Oct 2023 | 161 |
| Nov 2023 | 306 |
| Dec 2023 | 305 |
| Jan 2024 | 340 |
| Feb 2024 | 274 |
| Mar 2024 | 193 |
| Apr 2024 | 319 |
| May 2024 | 174 |
| Jun 2024 | 207 |
| Jul 2024 | 307 |
| Aug 2024 | 255 |
| Sep 2024 | 414 |
| Oct 2024 | 591 |
| Nov 2024 | 412 |
| Dec 2024 | 352 |
| Jan 2025 | 469 |
| Feb 2025 | 323 |
| Mar 2025 | 493 |
| Apr 2025 | 549 |
| May 2025 | 327 |
| Jun 2025 | 139 |
| Jul 2025 | 244 |
| Aug 2025 | 260 |
| Sep 2025 | 282 |
| Oct 2025 | 571 |
| Nov 2025 | 955 |
| Dec 2025 | 579 |
| Jan 2026 | 384 |
| Feb 2026 | 288 |
| Mar 2026 | 317 |
| Apr 2026 | 289 |
| May 2026 | 447 |
| Jun 2026 | 365 |
| Jul 2026 | 172 |
| Aug 2026 | 281 |
| Sep 2026 | 163 |
| Oct 2026 | 350 |
Popular in the same category
janomemocobeta922Japanese morphological analysis engine written in pure Python
mecab-python3SamuraiT581mecab-python. you can find original version here:http://taku910.github.io/mecab/
fugashipolm538A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.
sudachi.rsWorksApplications492SudachiPy 0.6* and above are developed as Sudachi.rs.
nagisataishi-i424A Japanese tokenizer based on recurrent neural networks
mecabikegami-yukino274This repository is for building Windows 64-bit MeCab binary and improving MeCab Python binding.
Resources in similar research fields
tokenizer-speed-benchlegalforce-research4Comparison code of various tokenizersShared: Responsible & Trustworthy NLP・Syntactic Text Processing
TinySegmenterMakershogo8214874A tool for creating a custom learning model for TinySegmenter.Shared: Syntactic Text Processing・Text Segmentation
nlp-recipes-jaupura66Samples codes for natural language processing in JapaneseShared: Syntactic Text Processing・Text Segmentation
react-native-japanese-tokenizercraftzdog40Async Japanese Tokenizer Native Plugin for React Native for iOS and AndroidShared: Syntactic Text Processing・Text Segmentation
WordCloud-Japaneseaocattleya10A script that enables morphological analysis-like display of Japanese sentences in WordCloud without using Mecab (a morphological analysis engine).Shared: Syntactic Text Processing・Text Segmentation
chirptextletuananh7ChirpText is a collection of text processing tools for Python.Shared: Syntactic Text Processing・Text Segmentation