
JMED-LLM: Japanese Medical Evaluation Dataset for Large Language Models
Description in other languages
Statistics
Statistics as of Nov 17, 2026
GitHub stars
59
±0 (Oct 13〜)
Last commit
Sep 2024
Created on GitHub
Jul 11, 2024
Listed
Jul 15, 2024
Stars over time
Show as a table
| Month | Stars |
|---|---|
| Dec 2025 | 53 |
| Jan 2026 | 55 |
| Feb 2026 | 56 |
| Mar 2026 | 56 |
| Apr 2026 | 57 |
| May 2026 | 57 |
| Jun 2026 | 59 |
| Jul 2026 | 59 |
| Aug 2026 | 59 |
| Sep 2026 | 59 |
| Oct 2026 | 59 |
| Nov 2026 | 59 |
Last snapshot of each month. Star counts of 1,000 or more are rounded to the nearest 100.
Popular in the same category
japanese-llm-benchmarkshi3z53A benchmark tool for evaluating Japanese language capabilities of various LLMs.
EDINET-BenchSakanaAI37[ICLR 2026] Evaluating the performance of LLMs on Japanese challenging financial tasks.
Japanese-RP-BenchAratako25Japanese-RP-Bench is a benchmark for measuring LLM's Japanese role-playing ability.
do-not-answer-jakunishou24A safety evaluation dataset "Do-Not-Answer" released by the University of Melbourne in August 2023 has been automatically translated into Japanese for use in the evaluation of Japanese LLM, and further modified to take into account Japanese culture.
jmle2026-benchnaoto-iwase11LLM benchmark on the 120th Japanese Medical Licensing Examination (Feb 7-8, 2026)
medLLM_QA_benchmarkaistairc6Trilingual (English, Japanese, Chinese) QA benchmark for medical LLM
Resources in similar research fields
- JMedBenchColdog23331kJMedBench is a Japanese biomedical LLM benchmark comprising 20 datasets across five tasks (MCQA, NER, STS, etc.) sourced from MedMCQA, PubMedQA, MMLU, and others, each with its own license, and includes a note that translations may contain biases requiring human review.HF datasetShared: Language Models・Semantic Text Processing
- japanese-gpt-neox-smallrinna656kA 12‑layer, 768‑hidden Japanese GPT‑NeoX model trained on CC‑100, C4, and Wikipedia, compatible with Huggingface, with an optional toy prefix‑tuning weight that forces each sentence to end with a smiling‑face emoji.HF modelShared: Language Models・Semantic Text Processing
- llm-jp-3-150mllm-jp75kLLM‑jp‑3‑150m—a 150‑M‑parameter Japanese language model from the National Institute of Informatics’ LLM R&D Center—is distributed in Hugging Face Transformers format, requiring torch ≥ 2.3.0, transformers ≥ 4.40.1, accelerate ≥ 0.29.3, flash‑attn ≥ 2.5.8, and was pretrained on Japanese Wikipedia, Common Crawl, WARP/PDF, WARP/HTML, and Kaken data using a unigram byte‑fallback tokenizer.HF modelShared: Language Models・Semantic Text Processing
- llm-jp-3-1.8bllm-jp3kA collection of Japanese large‑language models (1.8 b to 172 b beta1, with instruct variants) from NII’s R&D Center, packaged in Hugging Face Transformers format and pretrained on a mix of Japanese, English, and Web corpora totalling >1 trillion tokens, requiring torch ≥ 2.3, transformers ≥ 4.40, accelerate ≥ 0.29, and flash‑attn ≥ 2.5.HF modelShared: Language Models・Semantic Text Processing
- japanese-gpt-1brinna2kA 1.3‑B‑parameter, 24‑layer transformer GPT‑1B, trained on Japanese C4, CC‑100, and Wikipedia, was released by rinna Co. on January 26 2022 and is available under the MIT license.HF modelShared: Language Models・Semantic Text Processing
- deberta-v3-base-japaneseku-nlp13kJapanese DeBERTa V3 base pre‑trained on 540 B tokens from LLM‑jp v1.0, trained with a modified DeBERTa V3 setup, uses a unigram byte‑fallback tokenizer (no morphological analyzer), and is fine‑tuned for JGLUE NLU tasks.HF modelShared: Language Models・Semantic Text Processing