Explore resources

Combine keywords, purposes, tasks and categories. The filters stay in the URL, so you can share the view as is.

1,553 of 1,553

Aggregates over 150 GB of NicoNico Live’s comment logs from 2009‑2024—including pre‑transition, post‑transition, and real‑time NX‑Jikkyo captures—providing an API for easy retrieval of historical TV‑broadcast discussions.

Hugging Face downloads (last 30 days): 74kHugging Face likes: 102Datasets

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

GitHub stars: 167kLast commit: Nov 2026NLP pipelines

Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.

GitHub stars: 91kLast commit: Sep 2026OCR & document AI

opendatalab/MinerU

Multilingual

Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.

GitHub stars: 81kLast commit: Nov 2026OCR & document AI

A 12‑layer, 768‑hidden Japanese GPT‑NeoX model trained on CC‑100, C4, and Wikipedia, compatible with Huggingface, with an optional toy prefix‑tuning weight that forces each sentence to end with a smiling‑face emoji.

Hugging Face downloads (last 30 days): 656kHugging Face likes: 15Text generation

Dataset of dialogues and lore from the Fate/Stay Night character “Emilia”, formatted for training and evaluating conversational language models.

Hugging Face downloads (last 30 days): 18kHugging Face likes: 6Datasets

Ruri v3 is a state‑of‑the‑art Japanese text‑embedding model built on ModernBERT‑Ja, supporting up to 8,192‑token inputs, a 100K‑token vocabulary, FlashAttention‑accelerated inference, and multiple size variants for fast sentence‑transformer usage.

Hugging Face downloads (last 30 days): 634kHugging Face likes: 89Sentence similarity

milvus-io/milvus

Multilingual

Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search.

GitHub stars: 46kLast commit: Nov 2026Full-text search

ClimbLab‑Ja is a 300 billion‑token Japanese corpus, filtered from LLM‑jp Corpus v4 into 20 clusters for pre‑training language models, released under ODC‑BY with original source licenses and stored in parquet format.

Hugging Face downloads (last 30 days): 15kHugging Face likes: 2Datasets

Japanese‑adapted Spark‑X2.5‑4B model (Heretic JP) with refusal attenuation, pretraining, instruction tuning and tool‑calling repair, available in GGUF quantizations from BF16 to Q4_K_M (~2.5 GiB), usable via llama.cpp for 1,048,576‑token context.

Hugging Face downloads (last 30 days): 589kHugging Face likes: 22Text generation

OpenBMB/VoxCPM

Multilingual

VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning.

GitHub stars: 38kLast commit: Nov 2026Speech synthesis

hankcs/HanLP

Multilingual

Natural Language Processing for the next decade. Tokenization, Part-of-Speech Tagging, Named Entity Recognition, Syntactic & Semantic Dependency Parsing, Document Classification.

GitHub stars: 37kLast commit: Nov 2026NLP pipelines

Japanese Wikipedia‑based RAG dataset for the LLM course final competition, containing ~1 million text chunks and a search index.

Hugging Face downloads (last 30 days): 13kHugging Face likes: 37Datasets

Quantized GGUF versions of LiquidAI’s LFM 2.5‑Audio 1.5B JP model, including language, audio encoder, and vocoder weights (F32/F16/Q8_0/Q4_0), plus CLI/Server runners for ASR and TTS with llama.cpp.

Hugging Face downloads (last 30 days): 506kHugging Face likes: 37Audio to audio

qdrant/qdrant

Multilingual

Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/.

GitHub stars: 35kLast commit: Nov 2026Full-text search

explosion/spaCy

Multilingual

💫 Industrial-strength Natural Language Processing (NLP) in Python.

GitHub stars: 34kLast commit: Oct 2026NLP pipelines

JaidedAI/EasyOCR

Multilingual

Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.

GitHub stars: 30kLast commit: Nov 2026OCR & document AI

PLaMo‑Embedding‑1B is a Japanese text‑embedding model from Preferred Networks that converts Japanese text into vectors for information retrieval, classification, and clustering, shows strong performance on the JMTEB benchmark, and is freely available under an Apache v2.0 license.

Hugging Face downloads (last 30 days): 486kHugging Face likes: 49Sentence similarity

JP Voice‑Text Dataset for FGO Waifus: a 30,800‑record, 66.4‑hour Japanese audio collection of single‑voice‑actor character lines (≈7.76 s each) useful for ASR/ASV fine‑tuning and evaluation.

Hugging Face downloads (last 30 days): 10kHugging Face likes: 17Datasets

Open Source alternative to Algolia + Pinecone and an Easier-to-Use alternative to ElasticSearch ⚡ 🔍 ✨ Fast, typo tolerant, in-memory fuzzy Search Engine for building delightful search experiences.

GitHub stars: 27kLast commit: Nov 2026Full-text search

Japanese BERT‑base pretrained on 2019 Japanese Wikipedia with IPA‑dictionary and whole‑word masking, 12‑layer 768‑dim, 32,000‑vocab, 512‑token sequences, 1 M steps, available at cl‑tohoku/bert‑japanese under CC‑BY‑SA.

Hugging Face downloads (last 30 days): 462kHugging Face likes: 76Fill-mask

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

GitHub stars: 24kLast commit: May 2026Speech synthesis

FineWeb2 Edu Japanese delivers ~120 million high‑quality educational Japanese texts (≈89.3 billion tokens) from FineWeb2, filtered by a DeepSeek‑API classifier (score ≥ 2.5), tokenized via ModernBERT‑Ja‑130M, and includes a small‑token subset (≤512 tokens).

Hugging Face downloads (last 30 days): 8kHugging Face likes: 36Datasets