25.03.2026
Our paper CliniBench: A Clinical Outcome Prediction Benchmark for Generative and Encoder-Based Language Models by Paul Grundmann, Jan Frick, Dennis Fast, Thomas Steffek, Felix Gers, Wolfgang Nejdl and Alexander Löser was accepted at EACL 2026, the 19th Conference of the European Chapter of the Association for Computational Linguistics, in Rabat, Morocco. Jan Frick and Dennis Fast presented it in the poster session on 25 March 2026.
CliniBench is the first benchmark that compares generative large language models and encoder-based classifiers on the same task: predicting discharge diagnoses from admission notes in the MIMIC-IV dataset. Across 12 generative LLMs and 3 encoder-based classifiers, the encoders consistently come out ahead. Retrieval augmentation, which gives the LLMs similar patients as in-context examples, notably improves their results.
More background on the work and where it is heading is in the research project Predicting Clinical Outcomes. The benchmark is open source.
