How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

BenchmarkHub: an open-source platform for reproducible evaluation of large language models in healthcare

calendar_today November 13, 2026 person Alvarez-Arenas, J. I., Jimenez-Carretero, D., Sanchez-Cabo, F. domain medrxiv

Background: Large language models (LLMs) are solely judged by their scores on biomedical benchmarks, yet these scores are rarely reproducible since the prompts, inference settings and model builds are seldom reported. We developed BenchmarkHub, an open-source platform for auditable evaluation of language and vision-language models for biomedical research. Methods: BenchmarkHub is a Python web application (Django 5.2, SQLite) released under the MIT licence and available at https://github.com/juanjuanignacio/benchmarkhub.

open_in_new Read original post