Multimodal Reasoning

Advancing abnormality grounding via Vision-Language Models.

Bridging Modalities

Scientific Thesis

"Detecting anomalies is not sufficient for clinical decision-making. Current medical AI systems either detect visual deviations or describe images, but rarely connect the two through reasoning. This work reframes anomaly detection as a multimodal reasoning problem, where visual deviations are treated as hypotheses that must be grounded, explained, and contextualized through language and prior knowledge. We couple generative models with vision–language reasoning to enable systems that can both condition generation on semantic intent and interpret previously unseen anomalies in clinically meaningful terms. This closes the loop between detection, explanation, and reasoning, supporting open-world clinical settings."

Open Challenges

Multimodal Anomaly Grounding

Anchor visually detected anomalies in structured clinical language, ensuring that explanations correspond to anatomically and pathophysiologically valid concepts rather than free-form descriptions.

Language-Conditioned Generation and Repair

Develop generative models that can be steered by semantic constraints, enabling hypothesis-driven synthesis, counterfactual reasoning, and targeted normalization guided by clinical language.

Open-World Reasoning

Enable vision–language systems to reason about unseen anomalies, supporting interpretation of rare, ambiguous, or previously uncharacterized findings.

Key Publications

NOVA: A Benchmark for Rare Anomaly Localization and Clinical Reasoning in Brain MRI
NeurIPS 2025 (Oral) Full Manuscript

NOVA: A Benchmark for Rare Anomaly Localization and Clinical Reasoning in Brain MRI

Cosmin I. Bercea, Jun Li, Philipp Raffler, Evamaria O. Riedel, Lena Schmitzer, Angela Kurz, Felix Bitzer, Paula Roßmüller, Julian Canisius, Mirjam L. Beyrle, Che Liu, Wenjia Bai, Bernhard Kainz, Julia A. Schnabel, Benedikt Wiestler
Learning to Reason about Rare Diseases through Retrieval-Augmented Agents
arXiv 2025 Full Manuscript

Learning to Reason about Rare Diseases through Retrieval-Augmented Agents

Ha Young Kim, Jun Li, Ana Beatriz Solana, Carolin M. Pirkl, Benedikt Wiestler, Julia A. Schnabel*, Cosmin I. Bercea*
Enhancing Abnormality Grounding for Vision-Language Models with Knowledge Descriptions
WACV 2025 Full Manuscript

Enhancing Abnormality Grounding for Vision-Language Models with Knowledge Descriptions

Jun Li, Che Liu, Wenjia Bai, Rossella Arcucci, Cosmin I. Bercea*, Julia A. Schnabel*
Language Models Meet Anomaly Detection for Better Interpretability and Generalizability
MICCAI 2024 / MMMI Full Manuscript

Language Models Meet Anomaly Detection for Better Interpretability and Generalizability

Jun Li, Su Hwan Kim, Philip Müller, Lina Felsner, Daniel Rückert, Benedikt Wiestler, Julia A. Schnabel*, Cosmin I. Bercea*