Model Science · Explainable AI

Making AI systems
verifiable, explainable
and controllable.

I am a Full Professor at the University of Warsaw and serve as Director of the Centre for Credible AI at Warsaw University of Technology. I lead a research group working at the intersection of mathematics and computer science focused on deeper understanding of AI models.

My work asks a question benchmarks cannot answer: what is actually going on inside a model, and can we trust it where it will be deployed? Over the last decade this has meant new explanation methods, adversarial analysis of those methods, open-source tooling used by tens of thousands of practitioners, and a research programme I call Model Science.

AI is broken,
and we are here to fix it.
The research programme

Model Science

Read the agenda paper

Hundreds of millions of people now depend on models that essentially no one understands. Benchmarks saturate faster than new ones can be written, and a high leaderboard position tells you remarkably little about whether a system will hold up in a hospital, a courtroom, or a bank.

Model Science treats a trained model the way other mature disciplines treat their objects of study: as something to be probed, described, catalogued and reasoned about empirically. Cognitive science combines complementary analytical perspectives; neuroscience learns from deep single-case studies that population averages hide; medicine separates diagnosis, treatment and prevention into distinct but connected institutions; agriculture built shared infrastructure so that results accumulate instead of evaporating. AI research has none of this yet.

With Wojciech Samek (ECAI 2025) and in the fuller 2026 agenda paper, I organise the field around four questions.

01 Verify

Does the model perform reliably where it will actually be deployed?

Benchmarking is necessary and nowhere near sufficient. This pillar covers context-aware evaluation protocols, red teaming, auditing, robustness under distribution shift, and the systematic study of edge cases. Our work here includes red-teaming vision foundation models and analysing predictive multiplicity — the uncomfortable fact that many equally accurate models disagree on individual cases.

02 Explore

What computational mechanisms generate the output?

Attribution methods, concept-level analysis, sparse autoencoders, counterfactual explanations, and interactive tools that let a human actually interrogate a model. The framing matters: explanations exist to question a model, not to justify it after the fact — a position I have argued at ICML.

03 Steer

How do we change a trained model's behaviour without retraining it?

Once we can name what a model has internalised, we can intervene on it. Feature-level editing, representation steering and alignment techniques let us move a deployed system toward the behaviour we want — cheaply, reversibly, and with an audit trail.

04 Refine

Which training decisions produced this behaviour?

The loop closes here: connecting data, objectives and fine-tuning procedures to emergent misalignment, catastrophic forgetting and reward hacking. Understanding a model is only half the point if nothing flows back into how the next one is built.

The missing infrastructure

None of this scales without shared plumbing. Model Science calls for three things the field still lacks: dataset catalogues built on FAIR principles so that claims can be stress-tested across dozens of datasets rather than one; model catalogues that record lineage and inherited properties, because models are increasingly descendants rather than originals; and shared findings repositories — something like MITRE ATT&CK for model behaviour — so that a failure mode discovered once is known to everyone rather than rediscovered a hundred times.

Active research directions

Foundation models

Interpreting large multimodal models

Hierarchical sparse autoencoders that pull interpretable concepts out of CLIP-class models at a state-of-the-art sparsity/fidelity trade-off, and the question of what a "concept" in such a model even is.

XAI methods

Counterfactual explanation

Region-constrained visual counterfactuals, model-agnostic symbolic counterfactuals for time series, and global counterfactual directions — moving from "what mattered" to "what would have to change".

Robustness

Adversarial analysis of explanations

Explanations can be attacked. We showed partial dependence can be fooled by data poisoning, mapped the attack/defence landscape in a widely cited survey, and study when global feature effects can be trusted at all.

Medicine

Credible AI in healthcare

Interpretable time-to-event prediction, explanations for 3D segmentation models, radiomics for treatment-outcome prediction, and open clinical datasets — where a wrong model has a name attached to it.

Multiplicity

Rashomon sets & fairness

Many models fit the data equally well and behave very differently. We use that set — rather than a single winner — to reason about stability, bias and the real uncertainty behind a prediction.

Policy

Standards & the information ecosystem

Detection obligations for generative models, resistance against manipulative AI, and the attribution of ideas in AI research. Published in Nature Human Behaviour, Ethics and Information Technology, and at ECAI.

The MI².AI group

MI².AI brings together researchers from the mathematics and computer science faculties of the University of Warsaw and Warsaw University of Technology. The group is deliberately mixed, statisticians, ML researchers, clinicians' collaborators, because the questions do not respect departmental boundaries.

Hubert Baniecki PhD 2026 Paulina Tomaszewska PhD 2026 Katarzyna Woźnica PhD 2025 Katarzyna Kobylińska PhD 2025 Krzysztof Kotlarz PhD 2025 Weronika Hryniewska-Guzik PhD 2025 Barbara Rychalska PhD 2023 Tomasz Stanisławek PhD 2021
Publications

Selected publications

Full record on Scholar

A curated selection from 200+ publications, organised by theme rather than by count. The complete and continuously updated record lives on Google Scholar and DBLP.

Selection reflects methodological contribution and venue rather than citation count.

Open source

Methods that ship

drwhy.ai

A method that only exists in a paper is a method almost no one will use. The DrWhy.AI ecosystem is built on one idea: a model should be interrogable through a single, consistent interface regardless of what framework trained it. The design has been adopted well beyond our group — DALEX alone has been downloaded millions of times and is taught in courses on several continents.

DALEX

Model-agnostic explanations for R and Python

One explainer object, any model. Variable importance, partial dependence, break-down and SHAP attributions, residual diagnostics — the workhorse of the ecosystem. Published in JMLR.

survex

Explaining survival models

Time-dependent explanations for machine learning survival models, including SurvSHAP(t). Published in Bioinformatics; the reference tool for interpretable time-to-event modelling.

fairmodels

Bias detection, visualisation and mitigation

Checks a classifier against a battery of group-fairness criteria at once, and offers pre-, in- and post-processing mitigation. Published in The R Journal.

modelStudio

Interactive model exploration

Generates a serverless, shareable HTML dashboard from any explainer — the interface layer that turns explanation from a static plot into a conversation with the model.

Arena

Side-by-side model comparison

An interactive arena for comparing several models on the same observations — built for the Rashomon-set way of thinking, where the interesting question is where equally good models disagree.

archivist

Storing, restoring and searching R objects

Reproducibility infrastructure: every model, plot and dataset gets an address you can cite and retrieve years later.

Statistics & open data

Earlier packages, still maintained and still downloaded.

survminer

Drawing survival curves with ggplot2

My most-cited software contribution by a wide margin — a fixture of biomedical papers worldwide.

intsvy

International assessment data manager

Correct handling of plausible values and replicate weights for PISA, PIAAC, TIMSS and PIRLS — the studies that shape education policy.

ddst

Data-driven smooth tests

A family of goodness-of-fit tests that select their own complexity.

bgmm

Belief-based Gaussian mixture modelling

Mixture models that incorporate partial, uncertain label information.

PISA · PIAAC

OECD studies, ready to analyse

PISA 2000–2012 and PIAAC packaged for R, so that anyone can check what the headlines actually say.

Writing

Books

Beta & Bit series

Half of these are textbooks and half are comics, and that is deliberate. The people whose lives are shaped by predictive models — clinicians, teachers, regulators, students — are not going to read a NeurIPS paper. All titles are free to read online.

2025

We must explain! We will explain!

A comic journey through AI transparency. Beta and Bit show why post-hoc explanations matter — and where they mislead.

2023

The Hitchhiker's Guide to Responsible Machine Learning

Fifty-two pages, one story, and a complete reproducible ML workflow underneath it. Translated into several languages.

2023

Chart Runners

How to build plots that inform rather than impress — for audiences who did not ask for a chart in the first place.

2022

Chaos Game

Fractals and chaos theory, with worked examples in R, Python and Julia.

2022

Chasing Infinity

A comic about Zeno's paradoxes, infinite series, and the strange places where intuition fails.

2021

Explanatory Model Analysis

With Tomasz Burzykowski. A step-by-step treatment of the main methods of explainable machine learning; published by Chapman & Hall/CRC and free online.

2017

Przewodnik po pakiecie R

The Polish-language guide to R — basics, programming, statistics and visualisation. Now in its fourth edition.

2017

Eseje o sztuce wizualizacji danych

Essays on the design and aesthetics of data visualisation.

2015

Analiza danych z programem R

An academic textbook on estimation and hypothesis testing, worked through in R.

Teaching & talks

Teaching people to question models

Course repositories

Every course I run is built around the same conviction: students should leave able to break a model, not only to train one. Course materials — slides, code, projects — are public on GitHub, and the projects frequently turn into papers.

MSc / PhDUniversity of Warsaw

eXplainable Machine Learning

The flagship course. Students work through the modern XAI toolkit — attributions, counterfactuals, concept methods, sparse autoencoders — and then spend the second half of the semester attacking those methods on real models. Assessment is a written model audit, not an exam.

Materials on GitHub

MScSeminar

MSc Seminar: Machine Learning

Running continuously since 2017. A reading and research seminar where thesis projects are developed in public, critiqued by peers, and — often enough — pushed to a workshop or conference submission.

Seminar repository

Previously taught

CourseTopicYears
Data Visualisation TechniquesGrammar of graphics, perception, narrative visualisation2014–2019
Warsztaty Badawcze / Case StudiesEnd-to-end applied projects with external partners2015–2018
Advanced Programming and Data Analysis with RPackage development, performance, reproducibility2017–2018
Projekt Zespołowy / Team ProjectTeam-built data products2017–2019
Linear and Mixed ModelsFixed and random effects, applied inference2015–2017
Statistics II: Exploration, Prediction and VerificationStatistical learning foundations2015–2017
Programowanie i Wizualizacja z RR programming and visualisation2016–2018
R and Big DataOut-of-memory and distributed data analysis in R2014–2016
Data ProductsBuilding tools other people can use2015–2016

Talks & tutorials

I give roughly a dozen invited talks a year, to audiences that range from ML conferences to hospital boards to journalists. Three talks I am currently giving:

Model Science

Why benchmarks are not enough, what the four pillars are, and what infrastructure the field would need to make findings about models accumulate.

Keynote · 45 min

Explain to question, not to justify

A critical tour of what explanation methods actually deliver, how they can be attacked, and how to use them honestly in a high-stakes setting.

Invited talk / tutorial · 60–90 min

Credible AI in medicine

What goes wrong when a model meets a clinic — multiplicity, distribution shift, and the specific ways in which validation is usually too optimistic.

Domain audiences · 40 min

Prospective PhD students

Doing a PhD with me

Open positions at CCAI

I receive many more PhD enquiries than I can answer one by one, and a CV tells me very little about what it would be like to do research together. So before we talk about a PhD, I ask you to do one thing: a short, self-contained research task based on our recent work. It takes about two to four weeks of part-time work, and it is the best way for both of us to find out whether this kind of research suits you.

Who needs to do it? Everyone who has not worked with me before. If you have already co-authored a paper with our group, written a thesis under my supervision, or contributed to a project at the Centre for Credible AI, skip the task and simply email me.

The task in four steps

01 Pick

Which paper makes you want to argue with it?

Choose one paper from the list below and read it carefully, including the appendix. Pick the one that interests you most, not the one that looks easiest.

02 Reproduce

Can you get one key result to show up on your own machine?

Reproduce one central finding, at a smaller scale if needed: a smaller model, fewer samples, a public dataset. A laptop or free Colab is enough. Use the authors' code if it exists, but make sure you understand it.

03 Question

What would you have asked if you were a reviewer?

Formulate one question of your own and test it. Does the finding hold on another model, dataset or modality? Does it survive a change of hyperparameters? Is there a simpler baseline that does just as well?

04 Write

Could someone else repeat your experiment from your report?

Write a report of at most two pages (plus figures) and publish your code in a public repository. Structure: question, setup, results, what did not work, and what you would do next with three more months.

Papers to start from

All from the last two years of our group's work. The suggested questions are only starting points; your own question is better.

ExploreNeurons
LINE: LLM-based iterative neuron explanations for vision models
NeurIPS 2026 — V. Zaigrajew, M. Piechota, G. Sekula, P. Biecek

A black-box loop in which a language model proposes concepts and a text-to-image model tests them, to label what individual neurons respond to.

Possible question

How stable are the labels? Run the loop on a few neurons of a small vision model with different LLMs or seeds and measure how often they agree, and whether disagreement points to polysemantic neurons.

ExploreSparse autoencoders
Interpreting CLIP with hierarchical sparse autoencoders
ICML 2025 — V. Zaigrajew, H. Baniecki, P. Biecek

A hierarchical SAE architecture that extracts interpretable concepts from vision–language embeddings with a better sparsity/fidelity trade-off.

Possible question

Train a small SAE on CLIP embeddings of a public dataset twice with different seeds. Which concepts reappear and which do not? How much should we trust a single dictionary?

ExploreGame theory
The metagame of interpretability and meta-attributions
NeurIPS 2026 — H. Baniecki, P. Biecek, F. Fumagalli

Treats an attribution method itself as a cooperative game, to measure how one feature changes the attribution of another.

Possible question

On a small text or tabular model, find cases where meta-attributions change the top-k ranking of a standard attribution. How often does it happen, and can you predict when?

ExploreVision–language
Explaining similarity in vision–language encoders with weighted Banzhaf interactions
NeurIPS 2025 — H. Baniecki, M. Muschalik, F. Fumagalli, B. Hammer, E. Hüllermeier, P. Biecek

FIxLIP explains image–text similarity through cross-modal interactions instead of first-order saliency maps.

Possible question

Take captions involving counting or spatial relations, where CLIP-like models are known to fail. Do interaction explanations reveal the failure better than ordinary saliency maps?

VerifyMedicine
Auditing Sybil: explaining deep lung cancer risk prediction through generative interventional attributions
ICML 2026 — B. Sobieski, J. Grzywaczewski, K. Dobiczek, M. Wójcik, T. Bartczak, P. Szatkowski, P. Bombiński, M. Tivnan, P. Biecek

An interventional audit of a clinical model: realistic generative edits show what really drives the risk score, including failure modes.

Possible question

The full 3D pipeline is heavy, so transfer the idea: design a simple interventional audit of a public 2D medical classifier (insert or remove a structure) and compare it with what a saliency map suggests.

ExploreCounterfactuals
MASCOTS: model-agnostic symbolic counterfactual explanations for time series
ECML PKDD 2025 — D. Płudowski, F. Spinnato, P. Wilczyński, K. Kotowski, E.V. Ntagiou, R. Guidotti, P. Biecek

Counterfactuals built in a symbolic representation of time series, so that they are both faithful and readable.

Possible question

On a few datasets from the UCR archive, how does the choice of symbolic representation trade sparsity against plausibility? Would a domain expert find the counterfactuals useful?

Your own idea
Something else from Model Science
Any paper from the publication list, or a question of your own

If you have a strong idea about verifying, explaining or steering modern models, go for it, as long as it connects to our work and you can explain in two sentences why it matters.

Tip

Start from the Model Science agenda paper and look for a question in the gaps it describes.

What to send

One email with the subject line [PhD task] <paper> — <your name> and:

  • The report as a PDF: at most two pages plus figures — question, setup, results, what did not work, next steps.
  • A link to a public repository with code and a README that lets someone rerun your main experiment.
  • Your CV, including the expected date of your master's degree.
  • Three sentences on the research direction you would like to pursue and why.
  • A short note on tools: if you used AI assistants, say how. That is fine, but you must be able to explain every line and every decision.

What happens next

I read every report. If the work is promising, we schedule a 30–45 minute call, mostly about your experiment: why you made the choices you made, and what surprised you. A good report does not guarantee a position, but it is the start of every conversation about one.

Every candidate must also pass the regular admission to a doctoral school, at the University of Warsaw (admissions usually start in April) or Warsaw University of Technology (usually from July). A master's degree is required by the start of the PhD, so the best time to do the task is between autumn and early spring.

Frequently asked

Do I need publications?
No. A careful, honest report matters more than a list of papers.
What if my reproduction fails?
Then document why. A well-argued "this did not reproduce, and here is my best guess why" is a perfectly good report, and often a more interesting one.
Can I contact the authors?
Yes, with specific questions about the paper or the code. Please do not ask them to do the task for you.
Can I do an internship instead?
We do not have dedicated funding for internships. Open positions, when available, are listed on the CCAI website.
Is this also for postdocs?
No. If you already have a PhD, write to me directly with your research statement and two papers you are proud of.
Get in touch

Working together

I am interested in collaborations on the verification and interpretability of foundation models, and in applications where the stakes make credibility non-negotiable — medicine above all. I supervise doctoral students through MI².AI and, through Solutions42.ai, work with organisations that need existing models validated rather than new ones promised.

If you are considering a PhD in interpretability, verification or steering of modern models, start with the short research task described above; it is the first step before we talk. For talk invitations, please include the audience, date and format — there is no fee for public-interest and educational events.

Short biography

Przemysław Biecek is a Full Professor at the University of Warsaw and Director of the Centre for Credible AI at Warsaw University of Technology, where he leads the MI².AI research group. His work centres on Model Science — the interpretability, controllability and verifiability of machine learning systems. He has authored more than 200 scientific publications, is listed among the top 2% most-cited scientists worldwide, and is a Fulbright IMPACT Award laureate. His research appears in Nature Machine Intelligence, Nature Human Behaviour, NeurIPS, ICML, ICLR, CVPR, ECCV and MICCAI.

Alongside academic work he founded the Smarter Poland Foundation, which promotes data literacy in schools and public debate, and in 2021 founded Solutions42.ai after fifteen years of building solutions for universities and companies including Samsung, IBM and Disney.

Academic lineage: Witold Klonecki → Julian Perkal → Hugo Steinhaus → David Hilbert. Second (biological) advisor: Stanisław Cebrat.