Anita van Zwieten, Fiona M Blyth, Germaine Wong and Saman Khalatbari-Soltani
Epidemiologists are generally well equipped to design and conduct studies that minimise various types of bias, so as to obtain the most accurate estimates possible and therefore high-quality evidence. In observational studies, some types of bias, like confounding, have received a lot of attention, while others have been overlooked. One that has been neglected is overadjustment bias, which occurs when researchers adjust for an explanatory variable on the causal pathway from exposure to outcome when seeking to estimate the total effect.
Confounding occurs when a third variable that causes both the exposure and the outcome biases the estimated association. It is commonly dealt with by adjusting for potential confounders in the statistical models. Overadjustment bias often happens because researchers perceive adjustment as universally harmless or beneficial as a technique to deal with confounding. In reality, depending on the variables adjusted for and the underlying causal model, adjustment can be beneficial, have no impact, or even – as in the case of overadjustment – have detrimental impacts on the accuracy of estimates.
For instance, overadjustment is likely to result in bias towards the null, leading to an underestimation of the total effect. To illustrate this, researchers highlighted the impact that overadjustment would have on their total effect of interest (educational inequalities in health among people with chronic kidney disease) by building various models with different levels of adjustment and explicitly comparing the results. They showed that the relative risk of vascular events for people with no formal education, compared with those with a tertiary education, was reduced from 1.46 in their preferred model (confounder-adjusted only) to 1.15 in a model that also included mediators, including health behaviours, disease progression, and comorbidities.
There are also circumstances where overadjustment may lead to bias in any direction, such as when the adjusted variable is a collider – a variable that is caused by two or more variables through two or more unique causal paths.
Overadjustment is a common problem in many fields of epidemiology. As we have previously discussed in a primer, it is especially relevant in social epidemiology because of the complex, upstream and multifaceted pathways between social exposures and health outcomes. For example, overadjustment may occur if a researcher adjusts for health-related behaviours when trying to estimate the total effect of education on mortality (Figure 1). This is a problem because it is likely to lead to an underestimation of the effect of education on mortality.
Undertaking a systematic review of observational studies is a complex task that requires researchers to mitigate many potential sources of bias in the included studies, to ensure that their conclusions are robust enough to inform policy and practice decisions. Given the potential impact of overadjustment bias on study findings, we wondered how systematic reviewers navigate this.
In our scoping review published in IJE, we developed 12 criteria based on previous literature on overadjustment bias and used these to look at potential approaches to managing overadjustment bias in 84 systematic reviews of health inequalities. Overall, these approaches were not regularly applied. For instance, <5% of reviews clearly defined confounders and mediators, constructed causal diagrams, or considered overadjustment in their risk-of-bias assessment. In contrast, 54% included confounding in their risk-of-bias assessment.
Our findings are concerning, given the impact that underestimation of health inequalities could have on social and health policies, which in turn affect the lives of many people. We made practical recommendations that researchers from various disciplines can use to manage overadjustment and ensure it does not compromise review findings (Figure 2).
We wondered whether the limited consideration of overadjustment that we saw in systematic reviews might be due to a lack of awareness of this issue in the research community. So, we then investigated what relevant guidance reviewers have access to when conducting systematic reviews and meta-analyses of observational studies.
In our opinion piece also published in IJE, we reviewed 12 key risk-of-bias or critical appraisal tools (e.g. Quality in Prognosis Studies tool, ROBINS-I, ROBINS-E) and 10 key guidelines (e.g. Cochrane Handbook for Systematic Reviews of Interventions, Conducting Systematic Reviews and Meta-Analyses of Observational Studies of Etiology [COSMOS-E] and JBI Manual for Evidence Synthesis) for systematic reviews and meta-analyses of observational studies, to consider the extent to which they considered overadjustment bias and confounding bias. Only three more recent risk-of-bias tools (ROBINS-I, ROBINS-E and the Confounder Matrix) explicitly considered overadjustment. In contrast, all 12 of the tools explicitly considered confounding. None of the 10 guidelines gave explicit guidance on overadjustment bias, while four did for confounding bias.
We recommend that overadjustment bias be given explicit consideration in new revisions of guidelines for systematic reviews and meta-analyses. We also encourage review authors to adopt the more recent risk-of-bias tools, which include consideration of overadjustment.
More broadly, there is a need to raise awareness of the importance of balancing overadjustment and confounding biases when conducting primary studies and reviews. This requires judicious consideration of which variables are appropriate to adjust for in a given context. Sometimes there is no straightforward answer, but communicating transparently about our assumptions enables robust discussion and fosters high-quality evidence. These issues need to be highlighted not only in review guidelines and tools but also in epidemiological training, journal peer review, and publication processes to ensure that epidemiologists generate robust estimates that can be used effectively to improve the health of communities and tackle health inequalities.
Read more:
van Zwieten A, Dai J, Blyth FM, Wong G, Khalatbari-Soltani S. Overadjustment bias in systematic reviews and meta-analyses of socio-economic inequalities in health: a meta-research scoping review. Int J Epidemiol 2024; 53: dyad177
van Zwieten A, Blyth FM, Wong G, Khalatbari-Soltani S. Consideration of overadjustment bias in guidelines and tools for systematic reviews and meta-analyses of observational studies is long overdue. Int J Epidemiol 2024; 53: dyad174
Dr Anita van Zwieten (@anitavanzwieten) is a lecturer and social epidemiologist at the University of Sydney School of Public Health and the Centre for Kidney Research at Westmead. She has research expertise in life-course approaches to socioeconomic inequalities in health, health inequalities, and socioeconomic outcomes among people with chronic kidney disease, and methodological issues in social epidemiology.
Professor Fiona Blyth AM (@fionablyth2) is a professor of public health and pain medicine at the University of Sydney and an ARC Centre of Excellence in Population Ageing Research (CEPAR) Chief Investigator. She is a public health physician and pain epidemiologist who has been involved in studies of chronic pain epidemiology for almost 20 years, including large prospective cohort studies, randomised controlled trials, pharmacoepidemiological studies, and health services research using linked, routinely collected datasets.
Professor Germaine Wong (@germjacq) is the Director of Western Renal Service at Westmead Hospital, a professor of clinical epidemiology, NHMRC Leadership Fellow at the University of Sydney and Co-Director of Clinical Research at the Centre for Kidney Research. She has an internationally recognised track record in transplant epidemiology, cancer and transplantation, social ethics in organ allocation, decision analytical modelling, health economics, and quality-of-life studies in transplant recipients.
Dr Saman Khalatbari-Soltani (@saamaankh) is a social epidemiologist and senior lecturer in population health at the University of Sydney School of Public Health and CEPAR. Her research encompasses social determinants of health, healthy ageing, health inequalities, and the role of behavioural, psychological and biological factors in the genesis of health inequalities at older ages across the life course.
IJEblog 