Article

Free access

Reliable decision support using counterfactual models

Authors:

Suchi SariaAuthors Info & Claims

NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems

Pages 1696 - 1706

Published: 04 December 2017 Publication History

PDF eReader Publisher Site

Abstract

Decision-makers are faced with the challenge of estimating what is likely to happen when they take an action. For instance, if I choose not to treat this patient, are they likely to die? Practitioners commonly use supervised learning algorithms to fit predictive models that help decision-makers reason about likely future outcomes, but we show that this approach is unreliable, and sometimes even dangerous. The key issue is that supervised learning algorithms are highly sensitive to the policy used to choose actions in the training data, which causes the model to capture relationships that do not generalize. We propose using a different learning objective that predicts counterfactuals instead of predicting outcomes under an existing action policy as in supervised learning. To support decision-making in temporal settings, we introduce the Counterfactual Gaussian Process (CGP) to predict the counterfactual future progression of continuous-time trajectories under sequences of future actions. We demonstrate the benefits of the CGP on two important decision-support tasks: risk prediction and "what if?" reasoning for individualized treatment planning.

References

[1]

A.M. Alaa, J. Yoon, S. Hu, and M. van der Schaar. Personalized Risk Scoring for Critical Care Patients using Mixtures of Gaussian Process Experts. In ICML Workshop on Computational Frameworks for Personalization, 2016.

[2]

E. Arjas and J. Parner. Causal reasoning from longitudinal data. Scandinavian Journal of Statistics, 31(2):171-187, 2004.

[3]

L. Bottou, J. Peters, J.Q. Candela, D.X. Charles, M. Chickering, E. Portugaly, D. Ray, P.Y. Simard, and E. Snelson. Counterfactual reasoning and learning systems: the example of computational advertising. Journal of Machine Learning Research (JMLR), 14(1):3207-3260, 2013.

Digital Library

[4]

K.H. Brodersen, F. Gallusser, J. Koehler, N. Remy, and S.L. Scott. Inferring causal impact using bayesian structural time-series models. The Annals of Applied Statistics, 9(1):247-274, 2015.

[5]

R. Caruana, Y. Lou, J. Gehrke, P. Koch, M. Sturm, and N. Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In International Conference on Knowledge Discovery and Data Mining (KDD), pages 1721-1730. ACM, 2015.

Digital Library

[6]

L.F. Cheng, G. Darnell, C. Chivers, M.E. Draugelis, K. Li, and B.E. Engelhardt. Sparse multi-output Gaussian processes for medical time series prediction. arXiv preprint arXiv:1703.09112, 2017.

[7]

J. Cunningham, Z. Ghahramani, and C.E. Rasmussen. Gaussian processes for time-marked time-series data. In International Conference on Artificial Intelligence and Statistics (AISTATS), pages 255-263, 2012.

[8]

D.J. Daley and D. Vere-Jones. An Introduction to the Theory of Point Processes. Springer Science & Business Media, 2007.

[9]

S. Doroudi, P.S. Thomas, and E. Brunskill. Importance sampling for fair policy selection. In Uncertainty in Artificial Intelligence (UAI), 2017.

[10]

M. Dudík, J. Langford, and L. Li. Doubly robust policy evaluation and learning. In International Conference on Machine Learning (ICML), 2011.

[11]

K. Dyagilev and S. Saria. Learning (predictive) risk scores in the presence of censoring due to interventions. Machine Learning, 102(3):323-348, 2016.

Digital Library

[12]

M. Gong, K. Zhang, T. Liu, D. Tao, C. Glymour, and B. Schölkopf. Domain adaptation with conditional transferable components. In International Conference on Machine Learning (ICML), 2016.

[13]

A.G. Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, pages 83-90, 1971.

[14]

N. Jiang and L. Li. Doubly robust off-policy value evaluation for reinforcement learning. In International Conference on Machine Learning (ICML), pages 652-661, 2016.

[15]

F.D. Johansson, U. Shalit, and D. Sontag. Learning representations for counterfactual inference. In International Conference on Machine Learning (ICML), 2016.

Digital Library

[16]

H.L Li-wei, R.P. Adams, L. Mayaud, G.B. Moody, A. Malhotra, R.G. Mark, and S. Nemati. A physiological time series dynamics-based approach to patient monitoring and outcome prediction. IEEE Journal of Biomedical and Health Informatics, 19(3):1068-1076, 2015.

[17]

J.J. Lok. Statistical modeling of causal effects in continuous time. The Annals of Statistics, pages 1464-1507, 2008.

[18]

J.M. Mooij, D. Janzing, and B. Schölkopf. From ordinary differential equations to structural causal models: the deterministic case. 2013.

[19]

S.L. Morgan and C. Winship. Counterfactuals and causal inference. Cambridge University Press, 2014.

[20]

S.A. Murphy. Optimal dynamic treatment regimes. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65(2):331-355, 2003.

[21]

I. Nahum-Shani, M. Qian, D. Almirall, W.E. Pelham, B. Gnagy, G.A. Fabiano, J.G. Waxmonsky, J. Yu, and S.A. Murphy. Q-learning: A data analysis method for constructing adaptive interventions. Psychological Methods, 17(4):478, 2012.

[22]

J. Neyman. Sur les applications de la théorie des probability aux experiences agricoles: Essai des principes. Roczniki Nauk Rolniczych, 10:1-51, 1923.

[23]

J. Neyman. On the application of probability theory to agricultural experiments. Statistical Science, 5(4):465-472, 1990.

[24]

A.Y. Ng, A. Coates, M. Diel, V. Ganapathi, J. Schulte, B. Tse, E. Berger, and E. Liang. Autonomous inverted helicopter flight via reinforcement learning. In Experimental Robotics IX, pages 363-372. Springer, 2006.

[25]

J. Nocedal and S.J. Wright. Numerical optimization 2nd, 2006.

[26]

C. Păduraru, D. Precup, J. Pineau, and G. Comănici. An empirical analysis of off-policy learning in discrete mdps. In Workshop on Reinforcement Learning, page 89, 2012.

[27]

J. Pearl. Causality: models, reasoning and inference. Cambridge University Press, 2009.

Digital Library

[28]

C.E. Rasmussen and C.K.I. Williams. Gaussian processes for machine learning. the MIT Press, 2006.

Digital Library

[29]

M.T. Ribeiro, S. Singh, and C. Guestrin. Why should i trust you?: Explaining the predictions of any classifier. In International Conference on Knowledge Discovery and Data Mining (KDD), pages 1135-1144. ACM, 2016.

Digital Library

[30]

J.M. Robins. A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Mathematical Modelling, 7(9-12):1393-1512, 1986.

[31]

J.M. Robins. Estimation of the time-dependent accelerated failure time model in the presence of confounding factors. Biometrika, 79(2):321-334, 1992.

[32]

J.M. Robins. Causal inference from complex longitudinal data. In Latent variable modeling and applications to causality, pages 69-117. Springer, 1997.

[33]

J.M. Robins and M.A. Hernán. Estimation of the causal effects of time-varying exposures. Longitudinal data analysis, pages 553-599, 2009.

[34]

J.M. Robins, A. Rotnitzky, and D.O. Scharfstein. Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models. In Statistical models in epidemiology, the environment, and clinical trials, pages 1-94. Springer, 2000.

[35]

D.B. Rubin. Bayesian inference for causal effects: The role of randomization. The Annals of statistics, pages 34-58, 1978.

[36]

M. Saeed, M. Villarroel, A.T. Reisner, G. Clifford, L.W. Lehman, G. Moody, T. Heldt, T.H. Kyaw, B. Moody, and R.G. Mark. Multiparameter intelligent monitoring in intensive care II (MIMIC-II): a public-access intensive care unit database. Critical Care Medicine, 39(5):952, 2011.

[37]

D. Scharfstein, A. McDermott, W. Olson, and F. Wiegand. Global sensitivity analysis for repeated measures studies with informative dropout: A fully parametric approach. Statistics in Biopharmaceutical Research, 6(4):338-348, 2014.

[38]

P. Schulam and S. Saria. A framework for individualizing predictions of disease trajectories by exploiting multi-resolution structure. In Advances in Neural Information Processing Systems (NIPS), pages 748-756, 2015.

[39]

A. Sokol and N.R. Hansen. Causal interpretation of stochastic differential equations. Electronic Journal of Probability, 19(100):1-24, 2014.

[40]

H. Soleimani, A. Subbaswamy, and S. Saria. Treatment-response models for counterfactual reasoning with continuous-time, continuous-valued interventions. In Uncertainty in Artificial Intelligence (UAI), 2017.

[41]

R.S. Sutton and A.G. Barto. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998.

Digital Library

[42]

A. Swaminathan and T. Joachims. Counterfactual risk minimization. In International Conference on Machine Learning (ICML), 2015.

[43]

S.L. Taubman, J.M. Robins, M.A. Mittleman, and M.A. Hernán. Intervening on risk factors for coronary heart disease: an application of the parametric g-formula. International Journal of Epidemiology, 38(6):1599-1611, 2009.

[44]

J. Taylor, W. Cumberland, and J. Sy. A stochastic model for analysis of longitudinal AIDS data. Journal of the American Statistical Association, 89(427):727-736, 1994.

[45]

J. Wiens, J. Guttag, and E. Horvitz. Patient risk stratification with time-varying parameters: a multitask learning approach. Journal of Machine Learning Research (JMLR), 17(209):1-23, 2016.

Digital Library

[46]

Y. Xu, Y. Xu, and S. Saria. A Bayesian nonparametric approach for estimating individualized treatment-response curves. In Machine Learning for Healthcare Conference (MLHC), pages 282-300, 2016.

Cited By

Berrevoets JJordon JBica IGimson Avan der Schaar MLarochelle HRanzato MHadsell RBalcan MLin H(2020)OrganITEProceedings of the 34th International Conference on Neural Information Processing Systems10.5555/3495724.3497406(20037-20050)Online publication date: 6-Dec-2020
https://dl.acm.org/doi/10.5555/3495724.3497406
Xuan JLu JZhang G(2019)A Survey on Bayesian Nonparametric LearningACM Computing Surveys10.1145/329104452:1(1-36)Online publication date: 25-Jan-2019
https://dl.acm.org/doi/10.1145/3291044
Lim BAlaa ASchaar M(2018)Forecasting treatment responses over time using recurrent marginal structural networksProceedings of the 32nd International Conference on Neural Information Processing Systems10.5555/3327757.3327849(7494-7504)Online publication date: 3-Dec-2018
https://dl.acm.org/doi/10.5555/3327757.3327849
Show More Cited By

Reliable decision support using counterfactual models
1. Computing methodologies

Recommendations

USING decision models to enable better irrigation Decision Support Systems
Highlights
- Summary of decision theory and relevance to irrigation Decision Support Systems.
- Review of three different, sophisticated, decision modelling systems.
- A simple irrigation decision modelled using each system for comparison.
- Pros ...
Abstract
Many attempts have been made to enhance irrigation decisions using Decision Support Systems (DSS). These have met with limited success for many reasons, one of which is well known: that DSS encode decision rules (waterbalances, financial models) ...
Semi-supervised partial label learning algorithm via reliable label propagation
Abstract
Partial label learning (PLL) is a weakly supervised learning method that is able to predict one label as the correct answer from a given candidate label set. In PLL, when all possible candidate labels are as signed to real-world training examples, ...
Counterfactual explanations in sequential decision making under uncertainty
NIPS '21: Proceedings of the 35th International Conference on Neural Information Processing Systems

Methods to find counterfactual explanations have predominantly focused on one-step decision making processes. In this work, we initiate the development of methods to find counterfactual explanations for decision making processes in which multiple, ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems

December 2017

7104 pages

ISBN:9781510860964

Publisher

Curran Associates Inc.

Red Hook, NY, United States

Publication History

Published: 04 December 2017

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
197
Total Downloads

Downloads (Last 12 months)99
Downloads (Last 6 weeks)13

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Berrevoets JJordon JBica IGimson Avan der Schaar MLarochelle HRanzato MHadsell RBalcan MLin H(2020)OrganITEProceedings of the 34th International Conference on Neural Information Processing Systems10.5555/3495724.3497406(20037-20050)Online publication date: 6-Dec-2020
https://dl.acm.org/doi/10.5555/3495724.3497406
Xuan JLu JZhang G(2019)A Survey on Bayesian Nonparametric LearningACM Computing Surveys10.1145/329104452:1(1-36)Online publication date: 25-Jan-2019
https://dl.acm.org/doi/10.1145/3291044
Lim BAlaa ASchaar M(2018)Forecasting treatment responses over time using recurrent marginal structural networksProceedings of the 32nd International Conference on Neural Information Processing Systems10.5555/3327757.3327849(7494-7504)Online publication date: 3-Dec-2018
https://dl.acm.org/doi/10.5555/3327757.3327849
Liu YGottesman ORaghu AKomorowski MFaisal ADoshi-Velez FBrunskill E(2018)Representation balancing MDPs for off-policy policy evaluationProceedings of the 32nd International Conference on Neural Information Processing Systems10.5555/3327144.3327189(2649-2658)Online publication date: 3-Dec-2018
https://dl.acm.org/doi/10.5555/3327144.3327189

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents