Ringvorlesung 2026 "Künstliche Intelligenz - quo vadis?"
“Measuring machine intelligence: how do we know what LLMs can and cannot do?”
Referent: Dr. Ilia Kuznetsov (TU Darmstadt) - Vortrag auf Englisch
Kurzbeschreibung:
Large language models have transformed the AI landscape and are now embedded in everyday life -- used by individuals, businesses, and governments alike. Understanding what these models can and cannot do is essential to ensure their safe, reliable, and aligned deployment. Yet the very properties that make LLMs powerful also make them difficult to evaluate. Given a model, how can we know its true capabilities and limitations? To what extent do LLM behaviors persist across different tasks and contexts? What counts as convincing evidence of capability -- or its absence? And which common shortcuts and misconceptions should we avoid when reasoning about LLMs? In this talk, I will examine core paradigms for evaluating LLMs, highlight the challenges of making robust claims about their behavior, and discuss promising directions for more reliable and meaningful assessment.