
Evaluating LLMs on the 119th Japanese Medical Licensing Examination
Description in other languages
Statistics
Statistics as of Nov 17, 2026
GitHub stars
10
Last commit
Oct 2026
Created on GitHub
Mar 30, 2025
Listed
Oct 25, 2026
Popular in the same category
llm-leaderboardwandb95Project of llm evaluation to Japanese tasks
japanese-llm-rankingyuzu-ai50This repository supports YuzuAI's Rakuda leaderboard of Japanese LLMs, which is a Japanese-focused analogue of LMSYS' Vicuna eval.
llm-jp-eval-mmllm-jp44This tool automatically evaluates Japanese multi-modal large language models across multiple datasets.
japanese-llm-roleplay-benchmarkoshizo42This repository was created to evaluate the performance of character role-playing in Japanese LLM.
gpt4-autoevalnorthern-system-service17Script for automatically evaluating language model responses using GPT-4.
jp-tl-benchshisa-ai6Anchored Pairwise LLM Evaluation for Bidirectional Japanese-English Translation
Resources in similar research fields
IgakuQAjungokasai52Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing ExaminationsShared: Large language models・Evaluation & benchmarks
japanese_llm_evallightblue-tech5A repo for evaluating Japanese LLMs ・ 日本語LLMを評価するレポShared: Large language models・Evaluation & benchmarks
pitchbenchshewiiii2Experimental Japanese pitch accent based LLM BenchmarkShared: Large language models・Evaluation & benchmarks- jhumanevalkogi-jwu731JHumanEval is a hand‑translated Japanese version of the HumanEval benchmark, providing 164 Python programming problems with parallel English and Japanese comments to evaluate Japanese‑LLM code generation while preserving the original English errors.HF datasetShared: Large language models・Evaluation & benchmarks
- EDINET-BenchSakanaAI662EDINET‑Bench is a Japanese financial benchmark that evaluates LLMs on tasks such as accounting fraud detection, earnings forecasting, and industry prediction using ten years of EDINET‑API disclosed reports, with construction and evaluation code provided and the dataset relicensed to PDL 1.0.HF datasetShared: Large language models・Evaluation & benchmarks
jmed-llmsociocom59JMED-LLM: Japanese Medical Evaluation Dataset for Large Language ModelsShared: Large language models・Evaluation & benchmarks