PTW | Faculty Dialogues: „Pitfalls in Effect Measurement“

The internal Faculty Dialogues at the Faculty of Psychotherapy Science invite colleagues to deliver short keynote lectures on current topics, which are then explored in greater depth through open discussion with members of the faculty. The aim is to engage collectively in reflection on key issues in teaching, research, and practice, and to develop fresh perspectives.

The fifth edition of the event series, held on 13 November 2025, brought together numerous colleagues to discuss „Pitfalls in Effect Measurement: Clinicians’ vs. Epidemiologists’ Illusion or What Counts as Change? Rethinking Effect Measurement in Psychotherapy“.

In his talk, Alfred Uhl explored why studies on addiction remission report conflicting results. He shows that clinical research often underestimates remission due to the “Clinician’s Illusion”, while epidemiological studies overestimate it because of regression to the mean. These statistical artefacts, though long recognised, are often overlooked, leading to misleading conclusions about treatment effects. Uhl calls for greater methodological care to distinguish genuine change from measurement bias in psychotherapy research.

Learn more about the lecture

Background

We encounter a wide range of conflicting findings regarding spontaneous remission in individuals with addiction, spanning from very low remission rates in those receiving treatment to exceptionally high remission rates over relatively short periods in untreated individuals.

Methods

To resolve these contradictions, several widely cited publications were analysed, and potential theoretical reasons for these discrepancies were explored (Uhl, 2024).

Results

Significant artifacts, readily explained by probability theory, account for why remission rates—whether treatment-induced or spontaneous—are systematically underestimated in clinical settings and systematically overestimated in epidemiological studies. The systematic underestimation is primarily due to a form of Berkson’s Paradox, which Cohen and Cohen (1984) described as the „Clinician’s Illusion.“ The overestimation is largely due to artificial regression to the mean (Campbell & Kenny, 1999). Although both artifacts have been well-known for decades, they are usually ignored when findings are interpreted. Furthermore, highly questionable assumptions have sometimes been when interpreting routine data (e.g., Winick, 1962; Robins et al., 1974).

Conclusion

These issues deserve much greater attention in empirical studies aimed at quantifying remission. Certain biases can be mitigated or even avoided through careful study design, while others can be partially or fully compensated for through careful data analysis. In studies reporting high remission rates, it is often challenging to assess whether and to what extent these reflect actual changes or are artifacts.

References

Many thanks to all who contributed to such an engaging and thought-provoking discussion!

Faculty of Psychotherapy Science
Sigmund Freud Private University Vienna
Freudplatz 1, 4th floor, 1020 Vienna
ptw@sfu.ac.at

Save the date: The next Faculty Dialogue with Prof. Kathrin Mörtl on “Case Vignettes and Data Sharing: What’s Actually Allowed?” is taking place on 10 December, 2025 at 1 pm.

zum Seitenanfang