JMMLU is a Japanese Massive Multitask Language Understanding Benchmark featuring 7,536 teacher‑crafted questions across 56 subjects, including professional medicine, psychology, accounting, philosophy, and diverse high‑school disciplines.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
1,000
Hugging Face likes
14
Listed
May 13, 2024
Downloads (30-day window) over time
Show as a table
| Month | DL (30d) |
|---|---|
| Dec 2025 | 548 |
| Jan 2026 | 321 |
| Feb 2026 | 544 |
| Mar 2026 | 780 |
| Apr 2026 | 676 |
| May 2026 | 910 |
| Jun 2026 | 480 |
| Jul 2026 | 402 |
| Aug 2026 | 762 |
| Sep 2026 | 2,000 |
| Oct 2026 | 1,000 |
| Nov 2026 | 1,000 |
Downloads over the 30 days before the last snapshot of each month.
Popular in the same category
- emilia-yodasTTS-AGI18kDataset of dialogues and lore from the Fate/Stay Night character “Emilia”, formatted for training and evaluating conversational language models.
- fgo_voices_jpdeepghs10kJP Voice‑Text Dataset for FGO Waifus: a 30,800‑record, 66.4‑hour Japanese audio collection of single‑voice‑actor character lines (≈7.76 s each) useful for ASR/ASV fine‑tuning and evaluation.
- JGLUEshunk0313kUpdated JGLUE dataset card and loading script for a Japanese NLP benchmark (created by Yahoo Japan and Waseda University) that covers text classification (MARC‑ja, JCoLA), sentence‑pair classification (JNLI), and QA (JSQuAD, JCommonsenseQA), with releases linked on GitHub and Hugging Face.
- JMTEBsbintuitions3kJMTEB is a Japanese text‑embedding benchmark featuring 5 tasks (clustering, classification, STS, retrieval, reranking) and 28 datasets, offering a one‑line evaluation script and inviting community contributions.
- JMMMUJMMMU3kJMMMU is a Japanese multimodal benchmark expanded over tenfold to 1,320 culturally diverse questions (720 culture‑agnostic, 600 culture‑specific) translated by native subject experts, now featuring a public leaderboard.
- JamC-QAsbintuitions2kJamC‑QA is a bilingual benchmark of multiple‑choice questions spanning eight Japanese‑culture and knowledge categories, with leaderboard metrics comparing state‑of‑the‑art models.
Resources in similar research fields
- jhumanevalkogi-jwu731JHumanEval is a hand‑translated Japanese version of the HumanEval benchmark, providing 164 Python programming problems with parallel English and Japanese comments to evaluate Japanese‑LLM code generation while preserving the original English errors.Shared: Multilinguality・Text Generation
small_parallel_enjaodashi9750k English-Japanese Parallel Corpus for Machine Translation Benchmark.GitHubShared: Multilinguality・Text Generation
JASSMao-KU16JASS: Japanese-specific Sequence to Sequence Pre-training for Neural Machine Translation (LREC2020) & Linguistically Driven Multi-Task Pre-Training for Low-Resource Neural Machine Translation (ACM TALLIP)GitHubShared: Multilinguality・Text Generation- databricks-dolly-15k-jakunishou1kAn automatically translated Japanese version of the databricks‑dolly‑15k dataset, licensed CC‑BY‑SA‑3.0 and last updated on 2023‑05‑11.Shared: Multilinguality・Text Generation
CourseraParallelCorpusMiningshyyhs15Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures TranslationGitHubShared: Multilinguality・Text Generation
open-mantra-datasetmantra-inc208Dataset introduced in the paper Towards Fully Automated Manga Translation presented in AAAI21GitHubShared: Multilinguality・Text Generation