
A framework for few-shot evaluation of autoregressive language models.
Description in other languages
Statistics
Statistics as of Nov 17, 2026
GitHub stars
1
Last commit
Jun 2023
Created on GitHub
Jun 19, 2023
Listed
Oct 25, 2026
Popular in the same category
JGLUEyahoojapan349JGLUE: Japanese General Language Understanding Evaluation
llm-jp-evalllm-jp175This tool is designed to automatically evaluate large-scale Japanese language models across multiple datasets.
jp-stabletree155JP Language Model Evaluation Harness
japanese-lm-fin-harnesspfnet-research80Japanese Language Model Financial Evaluation Harness
japanese-llm-rankingyuzu-ai50This repository supports YuzuAI's Rakuda leaderboard of Japanese LLMs, which is a Japanese-focused analogue of LMSYS' Vicuna eval.
japanese-llm-roleplay-benchmarkoshizo42This repository was created to evaluate the performance of character role-playing in Japanese LLM.
Resources in similar research fields
japanese-dialog-transformersnttcslab247Code for evaluating Japanese pretrained models provided by NTT Ltd.Shared: Responsible & Trustworthy NLP・Language Models
temporalNLI_datasettomo-vv2Jamp: Controlled Japanese Temporal Inference Dataset for Evaluating Generalization Capacity of Language ModelsShared: Responsible & Trustworthy NLP・Language Models
llm-jp-sftllm-jp66This repository contains the code for supervised fine-tuning of LLM-jp models.Shared: Language Models・Semantic Text Processing
evaluate_japanese_w2vshihono12script to evaluate pre-trained Japanese word2vec model on Japanese similarity datasetShared: Responsible & Trustworthy NLP・Language Models
swallow-evaluation-instructswallow-llm34Swallow Project Post-learning Large-scale Language Model Evaluation FrameworkShared: Language Models- modernbert-ja-310msbintuitions36kModernBERT‑Ja‑310M is a Japanese‑language BERT variant that blends local‑global attention with RoPE, trained on 4.09 T tokens of Japanese/English text, supports a 102 400‑word vocabulary, 8 192‑token sequences, and is optimized for Flash Attention 2.HF modelShared: Language Models・Semantic Text Processing