JamC‑QA‑V2 is an expanded Japanese knowledge benchmark with multiple‑choice questions across five sub‑categories, evaluated by exact‑match accuracy for various language models on culture, geography, history, law, healthcare, and regional identity topics.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
148
Hugging Face likes
2
Listed
Nov 17, 2026
Popular in the same category
- JGLUEshunk0313kUpdated JGLUE dataset card and loading script for a Japanese NLP benchmark (created by Yahoo Japan and Waseda University) that covers text classification (MARC‑ja, JCoLA), sentence‑pair classification (JNLI), and QA (JSQuAD, JCommonsenseQA), with releases linked on GitHub and Hugging Face.
- JamC-QAsbintuitions2kJamC‑QA is a bilingual benchmark of multiple‑choice questions spanning eight Japanese‑culture and knowledge categories, with leaderboard metrics comparing state‑of‑the‑art models.
- JMMMU-ProJMMMU1kJMMMU‑Pro is an image‑based Japanese multimodal benchmark created by generating visual questions with a realistic model and human verification, revealing that current open‑source LMMs perform poorly on it and offering a cost‑effective method for future VQA benchmark development.
- JMedBenchColdog23331kJMedBench is a Japanese biomedical LLM benchmark comprising 20 datasets across five tasks (MCQA, NER, STS, etc.) sourced from MedMCQA, PubMedQA, MMLU, and others, each with its own license, and includes a note that translations may contain biases requiring human review.
- JAMMEvalllm-jp830JAMMEval is a distilled benchmark of seven Japanese VQA datasets, refined with two rounds of human annotation to remove ambiguities and non‑visual questions, delivering reliable evaluation of vision‑language models on multimodal Japanese tasks.
- llm-jp-evalllm-book666Dataset card for the ja‑vicuna‑qa‑benchmark used in the book “Introduction to Large‑Scale LLM II” and created by llm‑jp‑eval for cross‑dataset Japanese LLM evaluation (Apache 2.0).
Resources in similar research fields
belebelefacebookresearch339Repo for the Belebele dataset, a massively multilingual reading comprehension dataset.MultilingualShared: Corpora & datasets・Evaluation & benchmarks
tydiqagoogle-research-datasets319TyDi QA contains 200k human-annotated question-answer pairs in 11 Typologically Diverse languages, written without seeing the answer and without the use of translation, and is designed for the training and evaluation of automatic question answering systems. This repository provides evaluation code and a baseline system for the dataset.MultilingualShared: Corpora & datasets・Evaluation & benchmarks
ml-mkqaapple-aiml-research198We introduce MKQA, an open-domain question answering evaluation set comprising 10k question-answer pairs aligned across 26 typologically diverse languages (260k question-answer pairs in total). The goal of this dataset is to provide a challenging benchmark for question answering quality across a wide set of languages. Please refer to our paper for details, MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering.MultilingualShared: Corpora & datasets・Evaluation & benchmarks
mintakaamazon-science118Dataset from the paper "Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering" (COLING 2022)MultilingualShared: Corpora & datasets・Evaluation & benchmarks
jemhopqaaiishii30JEMHopQA (Japanese Explainable Multi-hop Question Answering) is a Japanese multi-hop QA dataset that can evaluate internal reasoning.GitHubShared: Corpora & datasets・Evaluation & benchmarks- JA-VG-VQA-500SakanaAI520JA‑VG‑VQA‑500 is a 500‑sample subset of the Japanese Visual Genome VQA dataset, licensed CC BY 4.0, used to benchmark EvoVLM‑JP‑v1‑7B.Shared: Corpora & datasets・Evaluation & benchmarks