
This is a repository that summarizes the tokenizer related to LLM being developed at the LLM Study Group (LLM-jp).
Description in other languages
Statistics
Statistics as of Nov 17, 2026
GitHub stars
54
+1 (Oct 13〜)
Last commit
Sep 2026
Created on GitHub
Jul 13, 2023
Listed
Oct 23, 2023
Stars over time
Show as a table
| Month | Stars |
|---|---|
| Dec 2025 | 44 |
| Jan 2026 | 44 |
| Feb 2026 | 45 |
| Mar 2026 | 46 |
| Apr 2026 | 47 |
| May 2026 | 47 |
| Jun 2026 | 48 |
| Jul 2026 | 48 |
| Aug 2026 | 49 |
| Sep 2026 | 50 |
| Oct 2026 | 53 |
| Nov 2026 | 54 |
Last snapshot of each month. Star counts of 1,000 or more are rounded to the nearest 100.
Popular in the same category
japanese-llm-rankingyuzu-ai50This repository supports YuzuAI's Rakuda leaderboard of Japanese LLMs, which is a Japanese-focused analogue of LMSYS' Vicuna eval.
llm-jp-evalllm-jp175This tool is designed to automatically evaluate large-scale Japanese language models across multiple datasets.
llm-lora-classificationhppRC97llm-lora-classification
llm-jp-sftllm-jp66This repository contains the code for supervised fine-tuning of LLM-jp models.
Japanese-BPEEncodertanreinama42Japanese-BPEEncoder
japanese-llm-roleplay-benchmarkoshizo42This repository was created to evaluate the performance of character role-playing in Japanese LLM.
Resources in similar research fields
yomichanFooSoft1.1kJapanese pop-up dictionary extension for Chrome and Firefox.Shared: Multilinguality・Annotation and Dataset Development
jmdict-yomitanyomidevs332JMdict, JMnedict, KANJIDIC for Yomitan/Yomichan.Shared: Multilinguality・Annotation and Dataset Development
jamdictneocl172Python 3 library for manipulating Jim Breen's JMdict, KanjiDic2, JMnedict and kanji-radical mappingsShared: Multilinguality・Annotation and Dataset Development
elmo-japanesecl-tohoku4Elmo (in Japanese)Shared: Multilinguality・Vocabulary, Dictionary, and Language Input Method
aoveceggplants3Easy aozorabunko Word2Vec Builder - Word2Vec Builder and pre-built model for all books in the Aozora Bunko library.Shared: Annotation and Dataset Development・Vocabulary, Dictionary, and Language Input Method
japagenretrieva1Investigation of pseudo-learning data generation using LLM in Japanese language tasksShared: Annotation and Dataset Development・Vocabulary, Dictionary, and Language Input Method