A leaderboard that ranks large language models by their semantic accuracy in translating Japanese visual novels into English, using cosine similarity over 256 samples and reporting chrF scores for comparison with tools like Sugoi Translator, Google Translate, Naver Papago, and Alibaba Translate.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
857
Hugging Face likes
44
Listed
Oct 29, 2024
Downloads (30-day window) over time
Show as a table
| Month | DL (30d) |
|---|---|
| Dec 2025 | 3,000 |
| Jan 2026 | 156 |
| Feb 2026 | 196 |
| Mar 2026 | — |
| Apr 2026 | 209 |
| May 2026 | 624 |
| Jun 2026 | — |
| Jul 2026 | 101 |
| Aug 2026 | 237 |
| Sep 2026 | 397 |
| Oct 2026 | 2,000 |
| Nov 2026 | 857 |
Downloads over the 30 days before the last snapshot of each month.
Popular in the same category
- JMMMUJMMMU3kJMMMU is a Japanese multimodal benchmark expanded over tenfold to 1,320 culturally diverse questions (720 culture‑agnostic, 600 culture‑specific) translated by native subject experts, now featuring a public leaderboard.
- JMMMU-ProJMMMU1kJMMMU‑Pro is an image‑based Japanese multimodal benchmark created by generating visual questions with a realistic model and human verification, revealing that current open‑source LMMs perform poorly on it and offering a cost‑effective method for future VQA benchmark development.
- JMedBenchColdog23331kJMedBench is a Japanese biomedical LLM benchmark comprising 20 datasets across five tasks (MCQA, NER, STS, etc.) sourced from MedMCQA, PubMedQA, MMLU, and others, each with its own license, and includes a note that translations may contain biases requiring human review.
- JAMMEvalllm-jp830JAMMEval is a distilled benchmark of seven Japanese VQA datasets, refined with two rounds of human annotation to remove ambiguities and non‑visual questions, delivering reliable evaluation of vision‑language models on multimodal Japanese tasks.
- jhumanevalkogi-jwu731JHumanEval is a hand‑translated Japanese version of the HumanEval benchmark, providing 164 Python programming problems with parallel English and Japanese comments to evaluate Japanese‑LLM code generation while preserving the original English errors.
- llm-jp-evalllm-book666Dataset card for the ja‑vicuna‑qa‑benchmark used in the book “Introduction to Large‑Scale LLM II” and created by llm‑jp‑eval for cross‑dataset Japanese LLM evaluation (Apache 2.0).
Resources in similar research fields
- EDINET-BenchSakanaAI662EDINET‑Bench is a Japanese financial benchmark that evaluates LLMs on tasks such as accounting fraud detection, earnings forecasting, and industry prediction using ten years of EDINET‑API disclosed reports, with construction and evaluation code provided and the dataset relicensed to PDL 1.0.Shared: Large language models・Corpora & datasets
jmed-llmsociocom59JMED-LLM: Japanese Medical Evaluation Dataset for Large Language ModelsGitHubShared: Large language models・Corpora & datasets
japanese-llm-benchmarkshi3z53A benchmark tool for evaluating Japanese language capabilities of various LLMs.GitHubShared: Large language models・Corpora & datasets- KokushiMD-10humanalysis-square327KokushiMD‑10 is a multilingual benchmark for ten Japanese healthcare professions, offering text‑only and image‑based single‑choice, multiple‑choice, calculation, and fill‑in‑the‑blank questions with chain‑of‑thought explanations, available in separate Japanese, English, or mixed splits.Shared: Corpora & datasets・Evaluation & benchmarks
- Japanese-Heron-Benchturing-motors316Japanese‑Heron‑Bench is a Japanese VLM benchmark comprising 21 images from seven subcategories, each paired with 3‑category questions (Conversation, Detail, Complex) for a total of 102 queries under CC BY licenses.Shared: Corpora & datasets・Evaluation & benchmarks
cameraCyberAgentAILab26CAMERA (CyberAgent Multimodal Evaluation for Ad Text GeneRAtion) is the Japanese ad text generation dataset.GitHubShared: Corpora & datasets・Evaluation & benchmarks