Multilingual

EleutherAI / lm-evaluation-harness

A framework for few-shot evaluation of language models.

Description in other languages

Statistics

Statistics as of Nov 17, 2026

GitHub stars
14,000
±0 (Oct 13〜)
Last commit
Sep 2026
Created on GitHub
Aug 28, 2020
Listed
—
Listed since the page was created

Stars over time

13,00013,50014,00014,50015,000NovSepJulMayMarJan 202614,000
Show as a table
MonthStars
Dec 2025—
Jan 2026—
Feb 2026—
Mar 2026—
Apr 2026—
May 2026—
Jun 2026—
Jul 2026—
Aug 2026—
Sep 202614,000
Oct 202614,000
Nov 202614,000

Last snapshot of each month. Star counts of 1,000 or more are rounded to the nearest 100.

Use on PyPI

PyPI packages: lm-eval · Monthly figures to Oct 2026 (ClickPy)

Downloads, last 12 months
14,697,225
Share from Japan (last 12 months)
0.5%
PyPI packages that depend on it
107
32 as a required dependency

Monthly downloads on PyPI

01M2M202620252024202320222021202020191,328,084
Show as a table
Monthdownloads (month)
Jan 2019—
Feb 2019—
Mar 2019—
Apr 2019—
May 2019—
Jun 2019—
Jul 2019—
Aug 2019—
Sep 2019—
Oct 2019—
Nov 2019—
Dec 2019—
Jan 2020—
Feb 2020—
Mar 2020—
Apr 2020—
May 2020—
Jun 2020—
Jul 2020—
Aug 2020—
Sep 2020—
Oct 2020—
Nov 2020—
Dec 2020—
Jan 2021—
Feb 2021—
Mar 2021—
Apr 2021—
May 2021—
Jun 2021—
Jul 2021—
Aug 2021—
Sep 2021414
Oct 2021164
Nov 2021191
Dec 2021148
Jan 2022180
Feb 2022230
Mar 2022836
Apr 2022858
May 2022754
Jun 20221,937
Jul 20221,665
Aug 20222,182
Sep 20221,740
Oct 20221,480
Nov 20221,767
Dec 20222,223
Jan 20232,104
Feb 20232,344
Mar 20235,648
Apr 20239,387
May 202323,830
Jun 202351,009
Jul 202323,237
Aug 202320,170
Sep 202326,593
Oct 202395,959
Nov 2023124,910
Dec 2023154,547
Jan 2024179,053
Feb 2024176,653
Mar 2024204,908
Apr 2024276,469
May 2024221,429
Jun 2024188,895
Jul 2024241,841
Aug 2024244,215
Sep 2024292,226
Oct 2024307,713
Nov 2024208,222
Dec 2024217,916
Jan 2025303,413
Feb 2025290,015
Mar 2025489,848
Apr 2025485,166
May 2025527,341
Jun 2025564,942
Jul 2025640,563
Aug 2025733,267
Sep 2025719,530
Oct 2025701,817
Nov 2025579,400
Dec 2025540,308
Jan 2026769,633
Feb 20261,272,019
Mar 20261,256,972
Apr 20261,346,427
May 20261,545,164
Jun 20261,714,737
Jul 20261,572,996
Aug 20261,711,811
Sep 20261,059,674
Oct 20261,328,084

Statistics for Japanese NLP as a whole

Popular in the same category

Resources in similar research fields

Related slides and articles

Ones that name this resource in their titles, and ones close to its description

Learn from slides & articles