Provides a Japanese RAG benchmark across five industry domains—finance, telecom, manufacturing, public, retail—by publishing datasets, an automatic evaluation framework, and comparative results for models such as Claude 3.5‑Sonnet, GPT‑4o, and others.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
246
Hugging Face likes
52
Listed
Sep 23, 2024
Downloads (30-day window) over time
Show as a table
| Month | DL (30d) |
|---|---|
| Dec 2025 | 333 |
| Jan 2026 | 304 |
| Feb 2026 | 286 |
| Mar 2026 | 144 |
| Apr 2026 | 112 |
| May 2026 | 143 |
| Jun 2026 | 147 |
| Jul 2026 | 225 |
| Aug 2026 | 153 |
| Sep 2026 | 275 |
| Oct 2026 | 389 |
| Nov 2026 | 246 |
Downloads over the 30 days before the last snapshot of each month.
Popular in the same category
- JMTEBsbintuitions3kJMTEB is a Japanese text‑embedding benchmark featuring 5 tasks (clustering, classification, STS, retrieval, reranking) and 28 datasets, offering a one‑line evaluation script and inviting community contributions.
- wikipedia-passages-jawiki-embeddingshotchpotch731Japanese Wikipedia sentences are transformed into various embeddings and a FAISS index, offering a Hugging Face Space demo, conversion scripts, and evaluations of search, Q&A, and OpenAI text‑embedding‑3‑small for RAG; embeddings are OpenAI‑licensed, others CC‑BY‑SA‑4.0.
- JQaRAhotchpotch436A Japanese QA dataset for evaluating Retrieval‑Augmented Generation (RAG), built from JAQKET questions and Wikipedia passages with gold retrieval‑relevance labels, released on HuggingFace and GitHub and scored primarily by nDCG@10.
- msmarco-ja-hard-negativeshotchpotch408Japanese hard‑negative mining and evaluation on the MSMARCO‑JA dataset, detailing normalization, filtering, BAAI/BGE reranker selection, random sampling, SPLADE‑based ranking, and chi‑square analysis showing statistically significant higher positive rates than the original mMARCO(ja).
- Japanese-RAG-Generator-Benchmarkneoai-inc113Japanese RAG Generator Benchmark (J‑RAGBench) supplies a multi‑category QA dataset—covering Integration, Reasoning, Logical, Table, and Abstention—designed to evaluate Japanese RAG generators, built with human effort and GPT‑4.1, and released under CC BY‑SA 4.0.
- emilia-yodasTTS-AGI18kDataset of dialogues and lore from the Fate/Stay Night character “Emilia”, formatted for training and evaluating conversational language models.
Resources in similar research fields
- ELYZA-tasks-100elyza2kA 100‑sample Japanese instruction‑tuning evaluation dataset of annotated tasks—ranging from summarization correction and math reasoning to translation, creative generation, and user‑intent understanding—designed for manual or automatic 5‑point rating of fine‑tuned models.Shared: Annotation and Dataset Development
- qg_jaquadlmqg494Japanese JaQuAD, a subset of QG‑Bench, provides sentence‑ and paragraph‑level data with highlighted answer tokens for training Japanese question‑generation models, evaluated by BLEU4, METEOR, ROUGE‑L, BERTScore, and MoverScore.Shared: Annotation and Dataset Development
- zenz-v2.5-datasetMiwa-Keita439A 190 M‑pair JSONL dataset for Japanese kana‑to‑kanji conversion, featuring “left context – input – output” triples and pre‑trained models (medium, small, xsmall) plus the AJIMEE-Bench benchmark, released under CC BY‑SA 4.0 with subsets licensed from Wikipedia and Common Crawl.Shared: Annotation and Dataset Development
ajimee-benchazookey30AJIMEE-Bench (Advanced Japanese IME Evaluation Benchmark)GitHubShared: Annotation and Dataset Development- WildGuardTestJPsbintuitions258WildGuardTestJP is a Japanese‑language evaluation dataset of 1,725 samples, faithfully translated from WildGuardTest through a multi‑stage refinement pipeline (Seed‑X‑PPO‑7B, gpt‑oss‑120b, Qwen2.5‑72B‑Instruct, gemma‑3‑27b‑it) and released on Hugging Face under the ODC‑BY license.Shared: Annotation and Dataset Development
- bbh-japfnet213Japanese‑language BB‑Bench Hard dataset with evaluation questions (JSON Lines) and Chain‑of‑Thought prompts (YAML), translated via the PLaMo model.Shared: Annotation and Dataset Development