Portal ENSP - Escola Nacional de Saúde Pública Sergio Arouca Portal FIOCRUZ - Fundação Oswaldo Cruz

Cadernos de Saúde Pública

ISSN 1678-4464

37 nº.6

Rio de Janeiro, Junho 2021


QUESTÕES METODOLÓGICAS

O uso do valor de E para análise de sensibilidade em estudos epidemiológicos

Conceição Christina Rigo Vale, Nubia Karla de Oliveira Almeida, Renan Moritz Varnier Rodrigues de Almeida

http://dx.doi.org/10.1590/0102-311X00294720


  • Artigo
  • Autores
  • Comentários (0)
  • Informações Suplementares




RESUMO
O estudo ilustra o uso de um índice recém-desenvolvido para análise de sensibilidade, o valor de E, útil para embasar inferências causais em estudos epidemiológicos observacionais. O valor de E busca identificar a força mínima da associação entre um fator de confusão não medido e uma exposição/desfecho que seria necessária para explicar a associação observada enquanto não causal, definido como valor de E-value = RR + √RR (RR - 1), onde RR é a razão de riscos entre a exposição e o desfecho. Aqui, o valor de E é exemplificado através de dados observacionais de um estudo recém-publicado sobre a relação entre indicadores de adequação da assistência pré-natal e o baixo peso ao nascer enquanto desfecho. O valor de E variou entre 1,45 e 5,63, a depender da categoria e do índice de assistência pré-natal avaliado, com o valor máximo para a categoria “sem assistência pré-natal” do índice GINDEX e o mínimo para “assistência pré-natal intermediária” do índice APNCU. Na categoria “assistência pré-natal inadequada” (todos os índices), o valor de E variou entre 2,76 e 4,99 (índices GINDEX e APNCU, respectivamente). Isso indica que apenas associações fortes entre um fator de confusão e baixo peso ao nascer (aumento de risco maior que 400%) seria capaz de explicar plenamente a associação entre assistência pré-natal e baixo peso ao nascer. O valor de E é uma ferramenta útil e intuitiva para análise de sensibilidade, que pode ajudar a fortalecer as inferências causais em estudos epidemiológicos observacionais.

Medidas de Associação, Exposição, Risco ou Desfecho; Estudos Observacionais como Assunto; Avaliação de Resultados em Cuidados de Saúde


 

Introduction

Sensitivity analyses are commonly used in observational epidemiologic studies to quantify the robustness of an investigated association to unmeasured or uncontrolled confounders 1. Traditional sensitivity analysis methods estimate the strength of association between the unmeasured confounder and the outcome (RR UD ) and between the unmeasured confounder and the exposure (RR EU ). After specifying these associations, one can calculate the influence of a given pair RR UD and RR EU on the risk ratio between exposure and outcome (RR ED ) Figure 1. The confounding factor (B) - maximum relative amount by which unmeasured confounders could reduce an observed and - is calculated as follows 2:

 

 

Figure 1 Scheme of a traditional sensitivity analysis (adapted from VanderWeele & Ding 1.

 

Eq.: 1

By dividing the observed by B, one obtains the maximum value by which a set of confounding factors could alter the observed RR ED . However, some author express concern about the subjectivity underlying sensitivity parameters choice (RR UD and RR EU ) 1,2. These parameters also entail simplifications related to unmeasured confounders, such as being defined as a binary variable or requiring the assumption of a single confounder 3,4,5, which negatively impact the robustness of sensitivity analyses and the causal inferences sought in observational studies.

Seeking to develop a simple and intuitive tool that waves the need for strong assumptions, Ding & VanderWeele 2 proposed a new sensitivity analysis technique for observational studies - the E-value. This tool aims to determine the “...minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates1 (p. 268). The E-value can be calculated as shown in Equation 2,

Eq.: 2

Where RR is the risk ratio between exposure and outcome. The E-value is conditional on the measured covariates and calculated based on the risk ratio scale used in the analysis. When the effect measure is the odds ratio (OR) and the outcome is relatively rare (prevalence below 15% in the population), OR can be used in Equation 2, defining the following:

Eq.: 3

Equation 3 is also applicable to a confidence interval (CI) parameter. For cases where the lower limit (LL) of the CI is lower than or equal to “1”, the E-value is considered one; otherwise, the E-value is determined as follows:

Eq.: 4

For large E-values, the unmeasured confounder will need a considerable impact to fully explain the effect estimate. Conversely, small values​ indicate that little impact would already be able to explain the effect estimate, indicating weak causal relations between the study variables.

Next, we will illustrate the use of the E-value using observational data from a recently-published study on the relationship between indicators of prenatal care adequacy and the outcome low birthweight 6.

Methods

An observational study conducted by Vale et al. 6 used multiple logistic modeling to investigate low birthweight in 368,093 singleton term live births in Rio de Janeiro, Brazil, from 2015 to 2016. Box 1 summarizes the study covariate variables and prenatal care indexes. The E-value was used to determine the minimum strength of association between possible unmeasured confounders and the outcome capable of altering results interpretation. E-values were calculated based on the adjusted OR and the lower limit of the 95% confidence interval (95%CI) of each prenatal care adequacy index (only statistically significant categories). The “Adequate prenatal care” category was used as reference (OR = 1.00). Analyzes were performed using the R Studio v.1.2.5001 (http://www.r-project.org) and the SPSS v.23 (https://www.ibm.com/).

 

 

Tab.: 1
Box 1 Definition of prenatal care adequacy indexes and covariates used for predicting low birthweight. Rio de Janeiro, Brazil, 2015-2016.

 

Results

The estimated E-value ranged between 1.45 and 5.63 according to the category and index evaluated Table 1, showing the highest value for the “no prenatal care” category of the GINDEX index and the minimum value for “intermediate prenatal care” of the adequacy of prenatal care utilization (APNCU) index. For “inappropriate prenatal care” (all indexes), the E-value ranged between 2.76 (GINDEX) and 4.99 (APNCU).

 

 

Tab.: 2
Table 1 E-value parameters calculated as a function of the adjusted odds ratio (OR) in logistic regression models for predicting low birthweight by prenatal care adequacy indexes.

 

Discussion

Based on the E-value parameter, researchers were able to determine the minimum required association between unmeasured potential confounders and low birthweight for explaining the observed associations. For instance, E-value reached its maximum value (4.99) when APNCU (the most discriminatory index) was considered “inappropriate prenatal care,” and only very strong confounders/low birthweight associations would be able to explain the prenatal care vs. low birthweight association observed.

The E-value method allows these results to be contrasted with association values for known risk factors not included in an traditional analysis. Studies approaching smoking during pregnancy, for example, reported an adjusted OR ranging from 1.23 to 2.63 7,8,9,10,11 - lower than that found by E-value parameters regarding the “inadequate prenatal care” category of all evaluated indexes, except for that of Ciari Jr. et al. 12 and Kessner et al. 13. This suggests that, alone, smoking during pregnancy is not capable of explaining the prenatal care vs. low birthweight association observed, thus strengthening causal inferences.

Likewise, studies reported adjusted ORs for alcohol and drug use during pregnancy ranging between 1.04 and 1.68 9, so that similar causal inferences may be made: the consumption of alcohol and drugs during pregnancy per se also would not comprise confounding factors capable of fully explaining the effects estimate.

Limitations

Just as any new metric, the E-value could be potentially misused, and should only be applied provided that researchers have a clear understanding of its scope and limitations. The recent literature on parameters for E-value has pointed out the following caveats for its use 14,15,16,17,18:

(1) The E-value is strictly concerned with the impact of unobserved confounders, evaluating no other biases such as sample bias, selective reporting, or other design flaws. These factors should be considered when interpreting an E-value, so that a good study with a low E-value may produce more reliable results than poorly designed and controlled studies with a high E-value.

(2) The E-value may be less useful in the presence of multiple, possibly interacting unmeasured confounders, in which case “...one should perhaps question whether the data available are in fact adequate to get a reasonable estimate of the causal effect at all (...) it is perhaps time to leave that study data alone and pursue other data sources more adequate”, as stated by VanderWeele et al. 17 (p. 4).

(3) Another limitation concerns the assumption of the same value for the confounder x exposure and confounder x outcome association. When this is not the case, more complicated methods were developed for applying E-value-like metrics 1,2. Using the index under these circumstances is valid upon the assumption that the E-value is a heuristic filter for the total maximum effect of all unknown confounders 14,15,16.

Conclusion

Many are the available procedures for conducting sensitivity analyses. However, for being considered “...too complicated to describe in reports, (...) too difficult to present, occupy too much space” and given that “reviewers and editors were often unsympathetic and believed that they could not be understood”, as emphasized by VanderWeele et al. 16 (p. 131-2), these procedures are not commonly used. Before this scenario, our study illustrated the use of a recently developed sensitivity index: the E-value, an intuitive tool of easy implementation that assemble the toolbox for dealing with causality inferences in non-experimental settings. “Statistical significance” metrics such as the p-value determines the existence of possible relationships between exposure and outcome, but fails in addressing potential bias arising from unmeasured confounders - to which end the E-value could be used.

Acknowledgments

We thank the Brazilian Graduate Studies Coordinating Board (CAPES) for the financial support offered for this research (grant code 001).

References

1.   VanderWeele TJ, Ding P. Sensitivity analysis in observational research: Introducing the E-value. Ann Intern Med 2017; 167:268-74.
2.   Ding P, VanderWeele TJ. Sensitivity analysis without assumptions. Epidemiology 2016; 27:368-77.
3.   Bross IDJ. Spurious effects from an extraneous variable. J Chronic Dis 1966; 19:637-47.
4.   Schelesselman JJ. Assessing effects of confounding variables. Am J Epidemiol 1978; 108:3-8.
5.   Rosenbaum PR, Rubin DB. Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. J R Stat Soc Series B Stat Methodol 1983; 45:212-8.
6.   Vale CCR, Almeida NKO, Almeida RMVR. Association between pre-natal care adequacy indexes and low birth weight outcome. Rev Bras Ginecol Obstet 2021; in press.
7.   Bell JF, Zimmerman FJ. Selection bias in prenatal care use by Medicaid recipients. Matern Child Health J 2003; 7:239-52.
8.   Zambonato AMK, Pinheiro RT, Horta BL, Tomasi E. Risk factors for small-for-gestational age births among infants in Brazil. Rev Saúde Pública 2004; 38:24-9.
9.   Reichman NE, Teitler JO. Timing of enhanced prenatal care and birth outcomes in New Jersey's Health Start program. Matern Child Health J 2005; 9:151-8.
10.   Xaverius P, Alman C, Holtz L, Yarber L. Risk factors associated with very low birth weight in a large urban area, stratified by adequacy of prenatal care. Matern Child Health J 2016; 20:623-9.
11.   Xaverius PK, O'Reilly Z, Li A, Flick LH, Arnold LD. Smoking cessation and pregnancy: timing of cessation reduces or eliminates the effect on low birth weight. Matern Child Health J 2019; 23:1434-41.
12.   Ciari Jr. C, Santos JLF, Almeida PAM. Avaliação quantitativa de serviços de pré-natal. Rev Saúde Pública 1972; 6:361-70.
13.   Kessner DM, Singer J, Kalk CE, Schlesinger ER. Infant death: an analysis by maternal risk and health care. Washington DC: Institute of Medicine/National Academy of Sciences; 1973.
14.   Ioannidis JPA, Tan YJ, Blum MR. Limitations and misinterpretations of E-values for sensitivity analyses of observational studies. Ann Intern Med 2019; 170:108-11.
15.   Blum MR, Tan YJ, Ioannidis JPA. Use of E-values for addressing confounding in observational studies - an empirical assessment of the literature. Int J Epidemiol 2020; 49:1482-94.
16.   VanderWeele TJ, Mathur MB, Ding P. Correcting misinterpretations of the E-Value. Ann Intern Med 2019; 170:131-2.
17.   VanderWeele TJ, Ding P, Mathur MB. Technical considerations in the use of the E-value. J Causal Infer 2019; 7:20180007.
18.   VanderWeele TJ, Mathur MB. Commentary: developing best-practice guidelines for the reporting of E-values. Int J Epidemiol 2020; 49:1495-97.
19.   Ministério da Saúde. Portaria nº 569, de 1º de junho de 2000. Estabelece o Programa de Humanização no Pré-natal e Nascimento. Diário Oficial da União 2000; 8 jun.
20.   Takeda SMP. Avaliação de unidade de atenção primária: modificação dos indicadores de saúde e qualidade da atenção [Masters Thesis]. Pelotas: Universidade Federal de Pelotas; 1993.
21.   Coutinho T, Teixeira MTB, Dain S, Sayd JD, Coutinho LM. Adequação do processo de assistência pré-natal entre as usuárias do Sistema Único de Saúde em Juiz de Fora-MG. Rev Bras Ginecol Obstet 2003; 25:717-24.

CreativeCommons
This is an open-access article distributed under the terms of the Creative Commons Attribution License

 


Cadernos de Saúde Pública | Reports in Public Health

Rua Leopoldo Bulhões 1480 - Rio de Janeiro RJ 21041-210 Brasil

Secretaria Editorial +55 21 2598-2511.
cadernos@fiocruz.br

  • APOIO:

©2015 | Cadernos de Saúde Pública - Escola Nacional de Saúde Pública Sergio Arouca | Fundação Oswaldo Cruz. - Ministério da Saúde Governo Federal | Desenvolvido por Riocom Design