Article

Information markets vs. opinion pools: an empirical comparison

Authors:

Chao-Hsien Chu,

David M. PennockAuthors Info & Claims

EC '05: Proceedings of the 6th ACM conference on Electronic commerce

Pages 58 - 67

https://doi.org/10.1145/1064009.1064016

Published: 05 June 2005 Publication History

Abstract

In this paper, we examine the relative forecast accuracy of information markets versus expert aggregation. We leverage a unique data source of almost 2000 people's subjective probability judgments on 2003 US National Football League games and compare with the "market probabilities" given by two different information markets on exactly the same events. We combine assessments of multiple experts via linear and logarithmic aggregation functions to form pooled predictions. Prices in information markets are used to derive market predictions. Our results show that, at the same time point ahead of the game, information markets provide as accurate predictions as pooled expert assessments. In screening pooled expert predictions, we find that arithmetic average is a robust and efficient pooling function; weighting expert assessments according to their past performance does not improve accuracy of pooled predictions; and logarithmic aggregation functions offer bolder predictions than linear aggregation functions. The results provide insights into the predictive performance of information markets, and the relative merits of selecting among various opinion pooling methods.

References

[1]

http://us.newsfutures.com

[2]

http://www.biz.uiowa.edu/iem/

[3]

http://www.hsx.com/

[4]

http://www.ideosphere.com/fx/

[5]

http://www.probabilityfootball.com/

[6]

http://www.probabilitysports.com/

[7]

http://www.tradesports.com/

[8]

A. H. Ashton and R. H. Ashton. Aggregating subjective forecasts: Some empirical results. Management Science, 31:1499--1508, 1985.

Digital Library

[9]

R. P. Batchelor and P. Dua. Forecaster diversity and the benefits of combining forecasts. Management Science, 41:68--75, 1995.

Digital Library

[10]

N. Cesa-Bianchi, Y. Freund, D. Haussler, D. P. Helmbold, R. E. Schapire, and M. K. Warmuth. How to use expert advice. Journal of the ACM, 44(3):427--485, 1997.

Digital Library

[11]

K. Chen, L. Fine, and B. Huberman. Predicting the future. Information System Frontier, 5(1):47--61, 2003.

Digital Library

[12]

R. T. Clemen and R. L. Winkler. Combining probability distributions from experts in risk analysis. Risk Analysis, 19(2):187--203, 1999.

[13]

R. M. Cook. Experts in Uncertainty: Opinion and Subjective Probability in Science. Oxford University Press, New York, 1991.

[14]

A. L. Delbecq, A. H. Van de Ven, and D. H. Gustafson. Group Techniques for Program Planners: A Guide to Nominal Group and Delphi Processes. Scott Foresman and Company, Glenview, IL, 1975.

[15]

E. F. Fama. Efficient capital market: A review of theory and empirical work. Journal of Finance, 25:383--417, 1970.

[16]

R. Forsythe and F. Lundholm. Information aggregation in an experimental market. Econometrica, 58:309--47, 1990.

[17]

R. Forsythe, F. Nelson, G. R. Neumann, and J. Wright. Forecasting elections: A market alternative to polls. In T. R. Palfrey, editor, Contemporary Laboratory Experiments in Political Economy, pages 69--111. University of Michigan Press, Ann Arbor,MI, 1991.

[18]

R. Forsythe, F. Nelson, G. R. Neumann, and J. Wright. Anatomy of an experimental political stock market. American Economic Review, 82(5):1142--1161, 1992.

[19]

R. Forsythe, T. A. Rietz, and T. W. Ross. Wishes, expectations, and actions: A survey on price formation in election stock markets. Journal of Economic Behavior and Organization, 39:83--110, 1999.

[20]

S. French. Group consensus probability distributions: a critical survey. Bayesian Statistics, 2:183--202, 1985.

[21]

C. Genest. A conflict between two axioms for combining subjective distributions. Journal of the Royal Statistical Society, 46(3):403--405, 1984.

[22]

C. Genest. Pooling operators with the marginalization property. Canadian Journal of Statistics, 12(2):153--163, 1984.

[23]

C. Genest, K. J. McConway, and M. J. Schervish. Characterization of externally Bayesian pooling operators. Annals of Statistics, 14(2):487--501, 1986.

[24]

C. Genest and J. V. Zidek. Combining probability distributions: A critique and an annotated bibliography. Statistical Science, 1(1):114--148, 1986.

[25]

S. J. Grossman. An introduction to the theory of rational expectations under asymmetric information. Review of Economic Studies, 48(4):541--559, 1981.

[26]

F. A. Hayek. The use of knowledge in society. American Economic Review, 35(4):519--530, 1945.

[27]

J. C. Jackwerth and M. Rubinstein. Recovering probability distribution from options prices. Journal of Finance, 51(5):1611--1631, 1996.

[28]

H. A. Linstone and M. Turoff. The Delphi Method: Techniques and Applications. Addison-Wesley, Reading, MA, 1975.

[29]

P. A. Morris. Decision analysis expert use. Management Science, 20(9):1233--1241, 1974.

[30]

P. A. Morris. Combining expert judgments: A bayesian approach. Management Science, 23(7):679--693, 1977.

Digital Library

[31]

P. A. Morris. An axiomatic approach to expert resolution. Management Science, 29(1):24--32, 1983.

Digital Library

[32]

E. W. Noreen. Computer-Intensive Methods for Testing Hypotheses: An Introduction. Wiley and Sons, Inc., New York, 1989.

[33]

D. M. Pennock, S. Lawrence, C. L. Giles, and F. A. Nielsen. The real power of artificial markets. Science, 291:987--988, February 2002.

[34]

D. M. Pennock, S. Lawrence, F. A. Nielsen, and C. L. Giles. Extracting collective probabilistic forecasts from web games. In Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 174--183, San Francisco, CA, 2001.

Digital Library

[35]

C. Plott and S. Sunder. Rational expectations and the aggregation of diverse information in laboratory security markets. Econometrica, 56:1085--118, 1988.

[36]

E. Servan-Schreiber, J. Wolfers, D. M. Pennock, and B. Galebach. Prediction markets: Does money matter? Electronic Markets, 14(3):243--251, 2004.

[37]

M. Spann and B. Skiera. Internet-based virtual stock markets for business forecasting. Management Science, 49(10):1310--1326, 2003.

Digital Library

[38]

M. West. Bayesian aggregation. Journal of the Royal Statistical Society. Series A. General, 147(4):600--607, 1984.

[39]

R. L. Winkler. The consensus of subjective probability distributions. Management Science, 15(2):B61--B75, 1968.

Digital Library

[40]

J. Wolfers and E. Zitzewitz. Prediction markets. Journal of Economic Perspectives, 18(2):107--126, 2004.

Cited By

Beekman JWoodaman RBuede D(2020)A Review of Probabilistic Opinion Pooling Algorithms with Application to Insider Threat DetectionDecision Analysis10.1287/deca.2019.039917:1(39-55)Online publication date: 1-Mar-2020
https://dl.acm.org/doi/10.1287/deca.2019.0399
Wu KWheatley SSornette D(2020)Inefficiency and predictability in the Brexit Pound market: a natural experimentThe European Journal of Finance10.1080/1351847X.2020.180578127:3(239-259)Online publication date: 13-Aug-2020
https://doi.org/10.1080/1351847X.2020.1805781
Winkler RGrushka-Cockayne YLichtendahl KJose V(2019)Probability Forecasts and Their CombinationDecision Analysis10.1287/deca.2019.039116:4(239-260)Online publication date: 1-Dec-2019
https://dl.acm.org/doi/10.1287/deca.2019.0391
Show More Cited By

Index Terms

Information markets vs. opinion pools: an empirical comparison
1. Applied computing
  1. Law, social and behavioral sciences
    1. Economics

Recommendations

Information transparency in prediction markets

Prediction markets are designed and conducted for the primary purpose of aggregating information so that market prices forecast future events. In such markets, a group of traders buy and sell contracts and the payoff depends on unknown future events. ...
Forecasting stock returns based on information transmission across global markets using support vector machines

This paper provides evidence that forecasts based on global stock returns transmission yield better returns in day trading, for both developed and emerging stock markets. The study investigates the performance of global stock market price transmission ...
Modeling volatility in prediction markets
EC '09: Proceedings of the 10th ACM conference on Electronic commerce

There is significant experimental evidence that prediction markets are efficient mechanisms for aggregating information and are more accurate in forecasting events than traditional forecasting methods, such as polls. Interpretation of prediction market ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EC '05: Proceedings of the 6th ACM conference on Electronic commerce

June 2005

302 pages

ISBN:1595930493

DOI:10.1145/1064009

General Chair:
John Riedl
University of Minnesota
,
Program Chairs:
Michael Kearns
University of Pennsylvania
,
Michael Reiter
Carnegie Mellon University, Pittsburgh, PA

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

EC05

Sponsor:

EC05: Sixth ACM Conference on Electronic Commerce 2005

June 5 - 8, 2005

BC, Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 664 of 2,389 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
615
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Beekman JWoodaman RBuede D(2020)A Review of Probabilistic Opinion Pooling Algorithms with Application to Insider Threat DetectionDecision Analysis10.1287/deca.2019.039917:1(39-55)Online publication date: 1-Mar-2020
https://dl.acm.org/doi/10.1287/deca.2019.0399
Wu KWheatley SSornette D(2020)Inefficiency and predictability in the Brexit Pound market: a natural experimentThe European Journal of Finance10.1080/1351847X.2020.180578127:3(239-259)Online publication date: 13-Aug-2020
https://doi.org/10.1080/1351847X.2020.1805781
Winkler RGrushka-Cockayne YLichtendahl KJose V(2019)Probability Forecasts and Their CombinationDecision Analysis10.1287/deca.2019.039116:4(239-260)Online publication date: 1-Dec-2019
https://dl.acm.org/doi/10.1287/deca.2019.0391
Canonico LFlathmann CMcNeese N(2019)The Wisdom of the Market: Using Human Factors to Design Prediction Markets for Collective IntelligenceProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118131963128263:1(1471-1475)Online publication date: 20-Nov-2019
https://doi.org/10.1177/1071181319631282
Sumari AAhmad A(2018)Cognitive Artificial Intelligence: Concept and Applications for HumankindIntelligent System10.5772/intechopen.72764Online publication date: 29-Aug-2018
https://doi.org/10.5772/intechopen.72764
Lekwijit SSutivong D(2018)Optimizing the liquidity parameter of logarithmic market scoring rules prediction marketsJournal of Modelling in Management10.1108/JM2-06-2017-006613:3(736-754)Online publication date: 13-Aug-2018
https://doi.org/10.1108/JM2-06-2017-0066
Brown AReade JVaughan Williams L(2018)When are prediction market prices most informative?International Journal of Forecasting10.1016/j.ijforecast.2018.05.005Online publication date: Aug-2018
https://doi.org/10.1016/j.ijforecast.2018.05.005
Atanasov PRescober PStone ESwift SServan-Schreiber ETetlock PUngar LMellers B(2017)Distilling the Wisdom of CrowdsManagement Science10.1287/mnsc.2015.237463:3(691-706)Online publication date: 1-Mar-2017
https://dl.acm.org/doi/10.1287/mnsc.2015.2374
Sumari AAhmad A(2017)Knowledge-growing system: The origin of the cognitive artificial intelligence2017 6th International Conference on Electrical Engineering and Informatics (ICEEI)10.1109/ICEEI.2017.8312382(1-7)Online publication date: Nov-2017
https://doi.org/10.1109/ICEEI.2017.8312382
Carvalho A(2016)An Overview of Applications of Proper Scoring RulesDecision Analysis10.1287/deca.2016.033713:4(223-242)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1287/deca.2016.0337
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents