Dataset card for the ja‑vicuna‑qa‑benchmark used in the book “Introduction to Large‑Scale LLM II” and created by llm‑jp‑eval for cross‑dataset Japanese LLM evaluation (Apache 2.0).
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
666
Hugging Face likes
3
Listed
Sep 4, 2024
Downloads (30-day window) over time
Show as a table
| Month | DL (30d) |
|---|---|
| Dec 2025 | 457 |
| Jan 2026 | 109 |
| Feb 2026 | 172 |
| Mar 2026 | 126 |
| Apr 2026 | 150 |
| May 2026 | 173 |
| Jun 2026 | 164 |
| Jul 2026 | 308 |
| Aug 2026 | 226 |
| Sep 2026 | 145 |
| Oct 2026 | 259 |
| Nov 2026 | 666 |
Downloads over the 30 days before the last snapshot of each month.
Popular in the same category
- JMedBenchColdog23331kJMedBench is a Japanese biomedical LLM benchmark comprising 20 datasets across five tasks (MCQA, NER, STS, etc.) sourced from MedMCQA, PubMedQA, MMLU, and others, each with its own license, and includes a note that translations may contain biases requiring human review.
- JDocQA-Reasoningricoh-ai269A Japanese benchmark, JDocQA Reasoning Bench, released by Ricoh for the GENIAC project, evaluates LMMs on 1,286 reasoning questions over complex charts, tables, and diagrams using automated LLM-as‑Judge scoring.
- ChouBunSakanaAI103ChouBun is a Japanese-language long‑context benchmark for LLMs featuring two task types—extractive QA (wiki_qa, edinet_qa) and abstractive summarization (corp_sec_qa, corp_sec_sum)—built from Wikipedia articles, EDINET security reports, and corporate documents, with standardized THUDM/LongBench format and up to 32768‑token contexts.
- JGLUEshunk0313kUpdated JGLUE dataset card and loading script for a Japanese NLP benchmark (created by Yahoo Japan and Waseda University) that covers text classification (MARC‑ja, JCoLA), sentence‑pair classification (JNLI), and QA (JSQuAD, JCommonsenseQA), with releases linked on GitHub and Hugging Face.
- JamC-QAsbintuitions2kJamC‑QA is a bilingual benchmark of multiple‑choice questions spanning eight Japanese‑culture and knowledge categories, with leaderboard metrics comparing state‑of‑the‑art models.
- JMMMU-ProJMMMU1kJMMMU‑Pro is an image‑based Japanese multimodal benchmark created by generating visual questions with a realistic model and human verification, revealing that current open‑source LMMs perform poorly on it and offering a cost‑effective method for future VQA benchmark development.
Resources in similar research fields
- JAQKETkumapo327A Japanese open‑domain QA dataset, JAQKET, provides quiz‑style questions with multiple choice answers (v1.0) and free‑text answer generation (v2.0), containing 13 061 training and 271 validation examples for v1.0 and 2 154 training and 1 164 validation examples for v2.0.Shared: Natural Language Interfaces・Annotation and Dataset Development
medLLM_QA_benchmarkaistairc6Trilingual (English, Japanese, Chinese) QA benchmark for medical LLMGitHubShared: Annotation and Dataset Development- JaQuADSkelterLabsInc916JaQuAD is a 2022 Japanese QA dataset of 39,696 SQuAD‑style extractive pairs from Wikipedia, totaling 73.2 MB, that achieves 78.92 % F1 (63.38 % EM) when fine‑tuned with BERT‑Japanese.Shared: Natural Language Interfaces・Annotation and Dataset Development
- auto-wiki-qacl-nagoya747AutoWikiQA is the largest free Japanese QA dataset, providing over 2.3 million handcrafted‑filtered question–answer pairs automatically generated from Wikipedia with Swallow‑MX and LLMs (without rule‑based templates) to support knowledge‑teaching and retrieval‑augmented generation applications.Shared: Annotation and Dataset Development
- nri-fin-reasoningnri-ai707Japanese instruction dataset with 632,636 multi‑turn samples (~6.35 B tokens) and GPT‑OSS‑120b reasoning traces for open‑ended, math, writing, and MCQA tasks across 135 finance topics plus 20 general ones, designed to fine‑tune LLM reasoning in finance.Shared: Annotation and Dataset Development
- EDINET-BenchSakanaAI662EDINET‑Bench is a Japanese financial benchmark that evaluates LLMs on tasks such as accounting fraud detection, earnings forecasting, and industry prediction using ten years of EDINET‑API disclosed reports, with construction and evaluation code provided and the dataset relicensed to PDL 1.0.Shared: Annotation and Dataset Development