Statistical inference

Last updated June 18, 2024

Statistical inference is the process of using data analysis to infer properties of an underlying distribution of probability.^[1]Inferential statistical analysis infers properties of a population, for example by testing hypotheses and deriving estimates. It is assumed that the observed data set is sampled from a larger population.

Inferential statistics can be contrasted with descriptive statistics. Descriptive statistics is solely concerned with properties of the observed data, and it does not rest on the assumption that the data come from a larger population. In machine learning, the term inference is sometimes used instead to mean "make a prediction, by evaluating an already trained model";^[2] in this context inferring properties of the model is referred to as training or learning (rather than inference), and using a model for prediction is referred to as inference (instead of prediction); see also predictive inference.

Introduction

Statistical inference makes propositions about a population, using data drawn from the population with some form of sampling. Given a hypothesis about a population, for which we wish to draw inferences, statistical inference consists of (first) selecting a statistical model of the process that generates the data and (second) deducing propositions from the model.^[3]

Konishi & Kitagawa state, "The majority of the problems in statistical inference can be considered to be problems related to statistical modeling".^[4] Relatedly, Sir David Cox has said, "How [the] translation from subject-matter problem to statistical model is done is often the most critical part of an analysis".^[5]

The conclusion of a statistical inference is a statistical proposition.^[6] Some common forms of statistical proposition are the following:

a point estimate, i.e. a particular value that best approximates some parameter of interest;
an interval estimate, e.g. a confidence interval (or set estimate), i.e. an interval constructed using a dataset drawn from a population so that, under repeated sampling of such datasets, such intervals would contain the true parameter value with the probability at the stated confidence level;
a credible interval, i.e. a set of values containing, for example, 95% of posterior belief;
rejection of a hypothesis;^{[note 1]}
clustering or classification of data points into groups.

Models and assumptions

Any statistical inference requires some assumptions. A statistical model is a set of assumptions concerning the generation of the observed data and similar data. Descriptions of statistical models usually emphasize the role of population quantities of interest, about which we wish to draw inference.^[7] Descriptive statistics are typically used as a preliminary step before more formal inferences are drawn.^[8]

Degree of models/assumptions

Statisticians distinguish between three levels of modeling assumptions;

Fully parametric : The probability distributions describing the data-generation process are assumed to be fully described by a family of probability distributions involving only a finite number of unknown parameters.^[7] For example, one may assume that the distribution of population values is truly Normal, with unknown mean and variance, and that datasets are generated by 'simple' random sampling. The family of generalized linear models is a widely used and flexible class of parametric models.
Non-parametric : The assumptions made about the process generating the data are much less than in parametric statistics and may be minimal.^[9] For example, every continuous probability distribution has a median, which may be estimated using the sample median or the Hodges–Lehmann–Sen estimator, which has good properties when the data arise from simple random sampling.
Semi-parametric : This term typically implies assumptions 'in between' fully and non-parametric approaches. For example, one may assume that a population distribution has a finite mean. Furthermore, one may assume that the mean response level in the population depends in a truly linear manner on some covariate (a parametric assumption) but not make any parametric assumption describing the variance around that mean (i.e. about the presence or possible form of any heteroscedasticity). More generally, semi-parametric models can often be separated into 'structural' and 'random variation' components. One component is treated parametrically and the other non-parametrically. The well-known Cox model is a set of semi-parametric assumptions.^{[ citation needed ]}

Importance of valid models/assumptions

Whatever level of assumption is made, correctly calibrated inference, in general, requires these assumptions to be correct; i.e. that the data-generating mechanisms really have been correctly specified.

Incorrect assumptions of 'simple' random sampling can invalidate statistical inference.^[10] More complex semi- and fully parametric assumptions are also cause for concern. For example, incorrectly assuming the Cox model can in some cases lead to faulty conclusions.^[11] Incorrect assumptions of Normality in the population also invalidates some forms of regression-based inference.^[12] The use of any parametric model is viewed skeptically by most experts in sampling human populations: "most sampling statisticians, when they deal with confidence intervals at all, limit themselves to statements about [estimators] based on very large samples, where the central limit theorem ensures that these [estimators] will have distributions that are nearly normal."^[13] In particular, a normal distribution "would be a totally unrealistic and catastrophically unwise assumption to make if we were dealing with any kind of economic population."^[13] Here, the central limit theorem states that the distribution of the sample mean "for very large samples" is approximately normally distributed, if the distribution is not heavy-tailed.

Approximate distributions

Given the difficulty in specifying exact distributions of sample statistics, many methods have been developed for approximating these.

With finite samples, approximation results measure how close a limiting distribution approaches the statistic's sample distribution: For example, with 10,000 independent samples the normal distribution approximates (to two digits of accuracy) the distribution of the sample mean for many population distributions, by the Berry–Esseen theorem.^[14] Yet for many practical purposes, the normal approximation provides a good approximation to the sample-mean's distribution when there are 10 (or more) independent samples, according to simulation studies and statisticians' experience.^[14] Following Kolmogorov's work in the 1950s, advanced statistics uses approximation theory and functional analysis to quantify the error of approximation. In this approach, the metric geometry of probability distributions is studied; this approach quantifies approximation error with, for example, the Kullback–Leibler divergence, Bregman divergence, and the Hellinger distance.^[15]^[16]^[17]

With indefinitely large samples, limiting results like the central limit theorem describe the sample statistic's limiting distribution if one exists. Limiting results are not statements about finite samples, and indeed are irrelevant to finite samples.^[18]^[19]^[20] However, the asymptotic theory of limiting distributions is often invoked for work with finite samples. For example, limiting results are often invoked to justify the generalized method of moments and the use of generalized estimating equations, which are popular in econometrics and biostatistics. The magnitude of the difference between the limiting distribution and the true distribution (formally, the 'error' of the approximation) can be assessed using simulation.^[21] The heuristic application of limiting results to finite samples is common practice in many applications, especially with low-dimensional models with log-concave likelihoods (such as with one-parameter exponential families).

Randomization-based models

For a given dataset that was produced by a randomization design, the randomization distribution of a statistic (under the null-hypothesis) is defined by evaluating the test statistic for all of the plans that could have been generated by the randomization design. In frequentist inference, the randomization allows inferences to be based on the randomization distribution rather than a subjective model, and this is important especially in survey sampling and design of experiments.^[22]^[23] Statistical inference from randomized studies is also more straightforward than many other situations.^[24]^[25]^[26] In Bayesian inference, randomization is also of importance: in survey sampling, use of sampling without replacement ensures the exchangeability of the sample with the population; in randomized experiments, randomization warrants a missing at random assumption for covariate information.^[27]

Objective randomization allows properly inductive procedures.^[28]^[29]^[30]^[31]^[32] Many statisticians prefer randomization-based analysis of data that was generated by well-defined randomization procedures.^[33] (However, it is true that in fields of science with developed theoretical knowledge and experimental control, randomized experiments may increase the costs of experimentation without improving the quality of inferences.^[34]^[35]) Similarly, results from randomized experiments are recommended by leading statistical authorities as allowing inferences with greater reliability than do observational studies of the same phenomena.^[36] However, a good observational study may be better than a bad randomized experiment.

The statistical analysis of a randomized experiment may be based on the randomization scheme stated in the experimental protocol and does not need a subjective model.^[37]^[38]

However, at any time, some hypotheses cannot be tested using objective statistical models, which accurately describe randomized experiments or random samples. In some cases, such randomized studies are uneconomical or unethical.

Model-based analysis of randomized experiments

It is standard practice to refer to a statistical model, e.g., a linear or logistic models, when analyzing data from randomized experiments.^[39] However, the randomization scheme guides the choice of a statistical model. It is not possible to choose an appropriate model without knowing the randomization scheme.^[23] Seriously misleading results can be obtained analyzing data from randomized experiments while ignoring the experimental protocol; common mistakes include forgetting the blocking used in an experiment and confusing repeated measurements on the same experimental unit with independent replicates of the treatment applied to different experimental units.^[40]

Model-free randomization inference

Model-free techniques provide a complement to model-based methods, which employ reductionist strategies of reality-simplification. The former combine, evolve, ensemble and train algorithms dynamically adapting to the contextual affinities of a process and learning the intrinsic characteristics of the observations.^[41]^[42]

For example, model-free simple linear regression is based either on

a random design, where the pairs of observations $(X_{1},Y_{1}),(X_{2},Y_{2}),\cdots ,(X_{n},Y_{n})$ are independent and identically distributed (iid), or
a deterministic design, where the variables $X_{1},X_{2},\cdots ,X_{n}$ are deterministic, but the corresponding response variables $Y_{1},Y_{2},\cdots ,Y_{n}$ are random and independent with a common conditional distribution, i.e., $P\left(Y_{j}\leq y|X_{j}=x\right)=D_{x}(y)$ , which is independent of the index $j$ .

In either case, the model-free randomization inference for features of the common conditional distribution $D_{x}(.)$ relies on some regularity conditions, e.g. functional smoothness. For instance, model-free randomization inference for the population feature conditional mean, $\mu (x)=E(Y|X=x)$ , can be consistently estimated via local averaging or local polynomial fitting, under the assumption that $\mu (x)$ is smooth. Also, relying on asymptotic normality or resampling, we can construct confidence intervals for the population feature, in this case, the conditional mean, $\mu (x)$ .^[43]

Paradigms for inference

Different schools of statistical inference have become established. These schools—or "paradigms"—are not mutually exclusive, and methods that work well under one paradigm often have attractive interpretations under other paradigms.

Bandyopadhyay & Forster describe four paradigms: The classical (or frequentist) paradigm, the Bayesian paradigm, the likelihoodist paradigm, and the Akaikean-Information Criterion-based paradigm.^[44]

Frequentist inference

This paradigm calibrates the plausibility of propositions by considering (notional) repeated sampling of a population distribution to produce datasets similar to the one at hand. By considering the dataset's characteristics under repeated sampling, the frequentist properties of a statistical proposition can be quantified—although in practice this quantification may be challenging.

Examples of frequentist inference

Frequentist inference, objectivity, and decision theory

One interpretation of frequentist inference (or classical inference) is that it is applicable only in terms of frequency probability; that is, in terms of repeated sampling from a population. However, the approach of Neyman^[45] develops these procedures in terms of pre-experiment probabilities. That is, before undertaking an experiment, one decides on a rule for coming to a conclusion such that the probability of being correct is controlled in a suitable way: such a probability need not have a frequentist or repeated sampling interpretation. In contrast, Bayesian inference works in terms of conditional probabilities (i.e. probabilities conditional on the observed data), compared to the marginal (but conditioned on unknown parameters) probabilities used in the frequentist approach.

The frequentist procedures of significance testing and confidence intervals can be constructed without regard to utility functions. However, some elements of frequentist statistics, such as statistical decision theory, do incorporate utility functions.^{[ citation needed ]} In particular, frequentist developments of optimal inference (such as minimum-variance unbiased estimators, or uniformly most powerful testing) make use of loss functions, which play the role of (negative) utility functions. Loss functions need not be explicitly stated for statistical theorists to prove that a statistical procedure has an optimality property.^[46] However, loss-functions are often useful for stating optimality properties: for example, median-unbiased estimators are optimal under absolute value loss functions, in that they minimize expected loss, and least squares estimators are optimal under squared error loss functions, in that they minimize expected loss.

While statisticians using frequentist inference must choose for themselves the parameters of interest, and the estimators/test statistic to be used, the absence of obviously explicit utilities and prior distributions has helped frequentist procedures to become widely viewed as 'objective'.^[47]

Bayesian inference

The Bayesian calculus describes degrees of belief using the 'language' of probability; beliefs are positive, integrate into one, and obey probability axioms. Bayesian inference uses the available posterior beliefs as the basis for making statistical propositions.^[48] There are several different justifications for using the Bayesian approach.

Examples of Bayesian inference

Credible interval for interval estimation
Bayes factors for model comparison

Bayesian inference, subjectivity and decision theory

Many informal Bayesian inferences are based on "intuitively reasonable" summaries of the posterior. For example, the posterior mean, median and mode, highest posterior density intervals, and Bayes Factors can all be motivated in this way. While a user's utility function need not be stated for this sort of inference, these summaries do all depend (to some extent) on stated prior beliefs, and are generally viewed as subjective conclusions. (Methods of prior construction which do not require external input have been proposed but not yet fully developed.)

Formally, Bayesian inference is calibrated with reference to an explicitly stated utility, or loss function; the 'Bayes rule' is the one which maximizes expected utility, averaged over the posterior uncertainty. Formal Bayesian inference therefore automatically provides optimal decisions in a decision theoretic sense. Given assumptions, data and utility, Bayesian inference can be made for essentially any problem, although not every statistical inference need have a Bayesian interpretation. Analyses which are not formally Bayesian can be (logically) incoherent; a feature of Bayesian procedures which use proper priors (i.e. those integrable to one) is that they are guaranteed to be coherent. Some advocates of Bayesian inference assert that inference must take place in this decision-theoretic framework, and that Bayesian inference should not conclude with the evaluation and summarization of posterior beliefs.

Likelihood-based inference

Likelihood-based inference is a paradigm used to estimate the parameters of a statistical model based on observed data. Likelihoodism approaches statistics by using the likelihood function, denoted as $L(\theta |x)$ , quantifies the probability of observing the given data $x$ , assuming a specific set of parameter values $\theta$ . In likelihood-based inference, the goal is to find the set of parameter values that maximizes the likelihood function, or equivalently, maximizes the probability of observing the given data.

The process of likelihood-based inference usually involves the following steps:

Formulating the statistical model: A statistical model is defined based on the problem at hand, specifying the distributional assumptions and the relationship between the observed data and the unknown parameters. The model can be simple, such as a normal distribution with known variance, or complex, such as a hierarchical model with multiple levels of random effects.
Constructing the likelihood function: Given the statistical model, the likelihood function is constructed by evaluating the joint probability density or mass function of the observed data as a function of the unknown parameters. This function represents the probability of observing the data for different values of the parameters.
Maximizing the likelihood function: The next step is to find the set of parameter values that maximizes the likelihood function. This can be achieved using optimization techniques such as numerical optimization algorithms. The estimated parameter values, often denoted as ${\bar {y}}$ , are the maximum likelihood estimates (MLEs).
Assessing uncertainty: Once the MLEs are obtained, it is crucial to quantify the uncertainty associated with the parameter estimates. This can be done by calculating standard errors, confidence intervals, or conducting hypothesis tests based on asymptotic theory or simulation techniques such as bootstrapping.
Model checking: After obtaining the parameter estimates and assessing their uncertainty, it is important to assess the adequacy of the statistical model. This involves checking the assumptions made in the model and evaluating the fit of the model to the data using goodness-of-fit tests, residual analysis, or graphical diagnostics.
Inference and interpretation: Finally, based on the estimated parameters and model assessment, statistical inference can be performed. This involves drawing conclusions about the population parameters, making predictions, or testing hypotheses based on the estimated model.

AIC-based inference

The Akaike information criterion (AIC) is an estimator of the relative quality of statistical models for a given set of data. Given a collection of models for the data, AIC estimates the quality of each model, relative to each of the other models. Thus, AIC provides a means for model selection.

AIC is founded on information theory: it offers an estimate of the relative information lost when a given model is used to represent the process that generated the data. (In doing so, it deals with the trade-off between the goodness of fit of the model and the simplicity of the model.)

Other paradigms for inference

Minimum description length

The minimum description length (MDL) principle has been developed from ideas in information theory ^[49] and the theory of Kolmogorov complexity.^[50] The (MDL) principle selects statistical models that maximally compress the data; inference proceeds without assuming counterfactual or non-falsifiable "data-generating mechanisms" or probability models for the data, as might be done in frequentist or Bayesian approaches.

However, if a "data generating mechanism" does exist in reality, then according to Shannon's source coding theorem it provides the MDL description of the data, on average and asymptotically.^[51] In minimizing description length (or descriptive complexity), MDL estimation is similar to maximum likelihood estimation and maximum a posteriori estimation (using maximum-entropy Bayesian priors). However, MDL avoids assuming that the underlying probability model is known; the MDL principle can also be applied without assumptions that e.g. the data arose from independent sampling.^[51]^[52]

The MDL principle has been applied in communication-coding theory in information theory, in linear regression,^[52] and in data mining.^[50]

The evaluation of MDL-based inferential procedures often uses techniques or criteria from computational complexity theory.^[53]

Fiducial inference

Fiducial inference was an approach to statistical inference based on fiducial probability, also known as a "fiducial distribution". In subsequent work, this approach has been called ill-defined, extremely limited in applicability, and even fallacious.^[54]^[55] However this argument is the same as that which shows^[56] that a so-called confidence distribution is not a valid probability distribution and, since this has not invalidated the application of confidence intervals, it does not necessarily invalidate conclusions drawn from fiducial arguments. An attempt was made to reinterpret the early work of Fisher's fiducial argument as a special case of an inference theory using upper and lower probabilities.^[57]

Structural inference

Developing ideas of Fisher and of Pitman from 1938 to 1939,^[58] George A. Barnard developed "structural inference" or "pivotal inference",^[59] an approach using invariant probabilities on group families. Barnard reformulated the arguments behind fiducial inference on a restricted class of models on which "fiducial" procedures would be well-defined and useful. Donald A. S. Fraser developed a general theory for structural inference^[60] based on group theory and applied this to linear models.^[61] The theory formulated by Fraser has close links to decision theory and Bayesian statistics and can provide optimal frequentist decision rules if they exist.^[62]

Inference topics

The topics below are usually included in the area of statistical inference.

Statistical assumptions
Statistical decision theory
Estimation theory
Statistical hypothesis testing
Revising opinions in statistics
Design of experiments, the analysis of variance, and regression
Survey sampling
Summarizing statistical data

Predictive inference

Predictive inference is an approach to statistical inference that emphasizes the prediction of future observations based on past observations.

Initially, predictive inference was based on observable parameters and it was the main purpose of studying probability,^{[ citation needed ]} but it fell out of favor in the 20th century due to a new parametric approach pioneered by Bruno de Finetti. The approach modeled phenomena as a physical system observed with error (e.g., celestial mechanics). De Finetti's idea of exchangeability—that future observations should behave like past observations—came to the attention of the English-speaking world with the 1974 translation from French of his 1937 paper,^[63] and has since been propounded by such statisticians as Seymour Geisser.^[64]

Notes

↑ According to Peirce, acceptance means that inquiry on this question ceases for the time being. In science, all scientific theories are revisable.

Related Research Articles

The likelihood function is the joint probability mass of observed data viewed as a function of the parameters of a statistical model. Intuitively, the likelihood function $is the probability of observing data assuming is the actual parameter.$

Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available. Fundamentally, Bayesian inference uses prior knowledge, in the form of a prior distribution in order to estimate posterior probabilities. Bayesian inference is an important technique in statistics, and especially in mathematical statistics. Bayesian updating is particularly important in the dynamic analysis of a sequence of data. Bayesian inference has found application in a wide range of activities, including science, engineering, philosophy, medicine, sport, and law. In the philosophy of decision theory, Bayesian inference is closely related to subjective probability, often called "Bayesian probability".

In statistics, point estimation involves the use of sample data to calculate a single value which is to serve as a "best guess" or "best estimate" of an unknown population parameter. More formally, it is the application of a point estimator to the data to obtain a point estimate.

In statistics, interval estimation is the use of sample data to estimate an interval of possible values of a parameter of interest. This is in contrast to point estimation, which gives a single value.

Informally, in frequentist statistics, a confidence interval (CI) is an interval which is expected to typically contain the parameter being estimated. More specifically, given a confidence level $, a CI is a random interval which contains the parameter being estimated % of the time. The confidence level, degree of confidence or confidence coefficient represents the long-run proportion of CIs that theoretically contain the true value of the parameter; this is tantamount to the nominal coverage probability. For example, out of all intervals computed at the 95% level, 95% of them should contain the parameter's true value.$

Minimum Description Length (MDL) is a model selection principle where the shortest description of the data is the best model. MDL methods learn through a data compression perspective and are sometimes described as mathematical applications of Occam's razor. The MDL principle can be extended to other forms of inductive inference and learning, for example to estimation and sequential prediction, without explicitly identifying a single model of the data.

Bayesian statistics is a theory in the field of statistics based on the Bayesian interpretation of probability, where probability expresses a degree of belief in an event. The degree of belief may be based on prior knowledge about the event, such as the results of previous experiments, or on personal beliefs about the event. This differs from a number of other interpretations of probability, such as the frequentist interpretation, which views probability as the limit of the relative frequency of an event after many trials. More concretely, analysis in Bayesian methods codifies prior knowledge in the form of a prior distribution.

In statistical inference, specifically predictive inference, a prediction interval is an estimate of an interval in which a future observation will fall, with a certain probability, given what has already been observed. Prediction intervals are often used in regression analysis.

Mathematical statistics is the application of probability theory, a branch of mathematics, to statistics, as opposed to techniques for collecting statistical data. Specific mathematical techniques which are used for this include mathematical analysis, linear algebra, stochastic analysis, differential equations, and measure theory.

This glossary of statistics and probability is a list of definitions of terms and concepts used in the mathematical sciences of statistics and probability, their sub-disciplines, and related fields. For additional related terms, see Glossary of mathematics and Glossary of experimental design.

In Bayesian statistics, a credible interval is an interval used to characterize a probability distribution. It is defined such that an unobserved parameter value has a particular probability to fall within it. For example, in an experiment that determines the distribution of possible values of the parameter $, if the probability that lies between 35 and 45 is 0.95, then is a 95% credible interval.$

Bootstrapping is any test or metric that uses random sampling with replacement, and falls under the broader class of resampling methods. Bootstrapping assigns measures of accuracy to sample estimates. This technique allows estimation of the sampling distribution of almost any statistic using random sampling methods.

Fiducial inference is one of a number of different types of statistical inference. These are rules, intended for general application, by which conclusions can be drawn from samples of data. In modern statistical practice, attempts to work with fiducial inference have fallen out of fashion in favour of frequentist inference, Bayesian inference and decision theory. However, fiducial inference is important in the history of statistics since its development led to the parallel development of concepts and tools in theoretical statistics that are widely used. Some current research in statistical methodology is either explicitly linked to fiducial inference or is closely connected to it.

The foundations of statistics consist of the mathematical and philosophical basis for arguments and inferences made using statistics. This includes the justification for the methods of statistical inference, estimation and hypothesis testing, the quantification of uncertainty in the conclusions of statistical arguments, and the interpretation of those conclusions in probabilistic terms. A valid foundation can be used to explain statistical paradoxes such as Simpson's paradox, provide a precise description of observed statistical laws, and guide the application of statistical conclusions in social and scientific applications.

Frequentist inference is a type of statistical inference based in frequentist probability, which treats “probability” in equivalent terms to “frequency” and draws conclusions from sample-data by means of emphasizing the frequency or proportion of findings in the data. Frequentist inference underlies frequentist statistics, in which the well-established methodologies of statistical hypothesis testing and confidence intervals are founded.

Bayesian econometrics is a branch of econometrics which applies Bayesian principles to economic modelling. Bayesianism is based on a degree-of-belief interpretation of probability, as opposed to a relative-frequency interpretation.

Algorithmic inference gathers new developments in the statistical inference methods made feasible by the powerful computing devices widely available to any data analyst. Cornerstones in this field are computational learning theory, granular computing, bioinformatics, and, long ago, structural probability . The main focus is on the algorithms which compute statistics rooting the study of a random phenomenon, along with the amount of data they must feed on to produce reliable results. This shifts the interest of mathematicians from the study of the distribution laws to the functional properties of the statistics, and the interest of computer scientists from the algorithms for processing data to the information they process.

In statistical inference, the concept of a confidence distribution (CD) has often been loosely referred to as a distribution function on the parameter space that can represent confidence intervals of all levels for a parameter of interest. Historically, it has typically been constructed by inverting the upper limits of lower sided confidence intervals of all levels, and it was also commonly associated with a fiducial interpretation, although it is a purely frequentist concept. A confidence distribution is NOT a probability distribution function of the parameter of interest, but may still be a function useful for making inferences.

In Bayesian inference, the Bernstein–von Mises theorem provides the basis for using Bayesian credible sets for confidence statements in parametric models. It states that under some conditions, a posterior distribution converges in the limit of infinite data to a multivariate normal distribution centered at the maximum likelihood estimator with covariance matrix given by $, where is the true population parameter and is the Fisher information matrix at the true population parameter value:$

Likelihoodist statistics or likelihoodism is an approach to statistics that exclusively or primarily uses the likelihood function. Likelihoodist statistics is a more minor school than the main approaches of Bayesian statistics and frequentist statistics, but has some adherents and applications. The central idea of likelihoodism is the likelihood principle: data are interpreted as evidence, and the strength of the evidence is measured by the likelihood function. Beyond this, there are significant differences within likelihood approaches: "orthodox" likelihoodists consider data only as evidence, and do not use it as the basis of statistical inference, while others make inferences based on likelihood, but without using Bayesian inference or frequentist inference. Likelihoodism is thus criticized for either not providing a basis for belief or action, or not satisfying the requirements of these other schools.

References

Citations

↑ Upton, G., Cook, I. (2008) Oxford Dictionary of Statistics, OUP. ISBN 978-0-19-954145-4.
↑ "TensorFlow Lite inference". The term inference refers to the process of executing a TensorFlow Lite model on-device in order to make predictions based on input data.
↑ Johnson, Richard (12 March 2016). "Statistical Inference". Encyclopedia of Mathematics. Springer: The European Mathematical Society. Retrieved 26 October 2022.
↑ Konishi & Kitagawa (2008), p. 75.
↑ Cox (2006), p. 197.
↑ "Statistical inference - Encyclopedia of Mathematics". www.encyclopediaofmath.org. Retrieved 2019-01-23.
1 2 Cox (2006) page 2
↑ Evans, Michael; et al. (2004). Probability and Statistics: The Science of Uncertainty. Freeman and Company. p. 267. ISBN 9780716747420.
↑ van der Vaart, A.W. (1998) Asymptotic Statistics Cambridge University Press. ISBN 0-521-78450-6 (page 341)
↑ Kruskal 1988
↑ Freedman, D.A. (2008) "Survival analysis: An Epidemiological hazard?". The American Statistician (2008) 62: 110-119. (Reprinted as Chapter 11 (pages 169–192) of Freedman (2010)).
↑ Berk, R. (2003) Regression Analysis: A Constructive Critique (Advanced Quantitative Techniques in the Social Sciences) (v. 11) Sage Publications. ISBN 0-7619-2904-5
1 2 Brewer, Ken (2002). Combined Survey Sampling Inference: Weighing of Basu's Elephants. Hodder Arnold. p. 6. ISBN 978-0340692295.
1 2 Jörgen Hoffman-Jörgensen's Probability With a View Towards Statistics, Volume I. Page 399 ^{[ full citation needed ]}
↑ Le Cam (1986) ^{[ page needed ]}
↑ Erik Torgerson (1991) Comparison of Statistical Experiments, volume 36 of Encyclopedia of Mathematics. Cambridge University Press. ^{[ full citation needed ]}
↑ Liese, Friedrich & Miescke, Klaus-J. (2008). Statistical Decision Theory: Estimation, Testing, and Selection. Springer. ISBN 978-0-387-73193-3.
↑ Kolmogorov (1963, p.369): "The frequency concept, based on the notion of limiting frequency as the number of trials increases to infinity, does not contribute anything to substantiate the applicability of the results of probability theory to real practical problems where we have always to deal with a finite number of trials".
↑ "Indeed, limit theorems 'as $n$ tends to infinity' are logically devoid of content about what happens at any particular $n$ . All they can do is suggest certain approaches whose performance must then be checked on the case at hand." — Le Cam (1986) (page xiv)
↑ Pfanzagl (1994): "The crucial drawback of asymptotic theory: What we expect from asymptotic theory are results which hold approximately . . . . What asymptotic theory has to offer are limit theorems."(page ix) "What counts for applications are approximations, not limits." (page 188)
↑ Pfanzagl (1994) : "By taking a limit theorem as being approximately true for large sample sizes, we commit an error the size of which is unknown. [. . .] Realistic information about the remaining errors may be obtained by simulations." (page ix)
↑ Neyman, J.(1934) "On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection", Journal of the Royal Statistical Society , 97 (4), 557–625 JSTOR 2342192
1 2 Hinkelmann and Kempthorne(2008) ^{[ page needed ]}
↑ ASA Guidelines for the first course in statistics for non-statisticians. (available at the ASA website)
↑ David A. Freedman et alia's Statistics.
↑ Moore et al. (2015).
↑ Gelman A. et al. (2013). Bayesian Data Analysis (Chapman & Hall).
↑ Peirce (1877-1878)
↑ Peirce (1883)
↑ Freedman, Pisani & Purves 1978.
↑ David A. Freedman Statistical Models.
↑ Rao, C.R. (1997) Statistics and Truth: Putting Chance to Work, World Scientific. ISBN 981-02-3111-3
↑ Peirce; Freedman; Moore et al. (2015).^{[ citation needed ]}
↑ Box, G.E.P. and Friends (2006) Improving Almost Anything: Ideas and Essays, Revised Edition, Wiley. ISBN 978-0-471-72755-2
↑ Cox (2006), p. 196.
↑
ASA Guidelines for the first course in statistics for non-statisticians. (available at the ASA website)
- David A. Freedman et alias Statistics.
- Moore et al. (2015).
↑ Neyman, Jerzy. 1923 [1990]. "On the Application of Probability Theory to AgriculturalExperiments. Essay on Principles. Section 9." Statistical Science 5 (4): 465–472. Trans. Dorota M. Dabrowska and Terence P. Speed.
↑ Hinkelmann & Kempthorne (2008) ^{[ page needed ]}
↑ Dinov, Ivo; Palanimalai, Selvam; Khare, Ashwini; Christou, Nicolas (2018). "Randomization-based statistical inference: A resampling and simulation infrastructure". Teaching Statistics. 40 (2): 64–73. doi:10.1111/test.12156. PMC 6155997 . PMID 30270947.
↑ Hinkelmann and Kempthorne (2008) Chapter 6.
↑ Dinov, Ivo; Palanimalai, Selvam; Khare, Ashwini; Christou, Nicolas (2018). "Randomization-based statistical inference: A resampling and simulation infrastructure". Teaching Statistics. 40 (2): 64–73. doi:10.1111/test.12156. PMC 6155997 . PMID 30270947.
↑ Tang, Ming; Gao, Chao; Goutman, Stephen; Kalinin, Alexandr; Mukherjee, Bhramar; Guan, Yuanfang; Dinov, Ivo (2019). "Model-Based and Model-Free Techniques for Amyotrophic Lateral Sclerosis Diagnostic Prediction and Patient Clustering". Neuroinformatics. 17 (3): 407–421. doi:10.1007/s12021-018-9406-9. PMC 6527505 . PMID 30460455.
↑ Politis, D.N. (2019). "Model-free inference in statistics: how and why". IMS Bulletin. 48.
↑ Bandyopadhyay & Forster (2011). See the book's Introduction (p.3) and "Section III: Four Paradigms of Statistics".
↑ Neyman, J. (1937). "Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability". Philosophical Transactions of the Royal Society of London A. 236 (767): 333–380. Bibcode:1937RSPTA.236..333N. doi: 10.1098/rsta.1937.0005 . JSTOR 91337.
↑ Preface to Pfanzagl.
↑ Little, Roderick J. (2006). "Calibrated Bayes: A Bayes/Frequentist Roadmap". The American Statistician. 60 (3): 213–223. doi:10.1198/000313006X117837. ISSN 0003-1305. JSTOR 27643780. S2CID 53505632.
↑ Lee, Se Yoon (2021). "Gibbs sampler and coordinate ascent variational inference: A set-theoretical review". Communications in Statistics - Theory and Methods. 51 (6): 1549–1568. arXiv: 2008.01006 . doi:10.1080/03610926.2021.1921214. S2CID 220935477.
↑ Soofi (2000)
1 2 Hansen & Yu (2001)
1 2 Hansen and Yu (2001), page 747.
1 2 Rissanen (1989), page 84
↑ Joseph F. Traub, G. W. Wasilkowski, and H. Wozniakowski. (1988) ^{[ page needed ]}
↑ Neyman (1956)
↑ Zabell (1992)
↑ Cox (2006) page 66
↑ Hampel 2003.
↑ Davison, page 12. ^{[ full citation needed ]}
↑ Barnard, G.A. (1995) "Pivotal Models and the Fiducial Argument", International Statistical Review, 63 (3), 309–323. JSTOR 1403482
↑ Fraser, D. A. S. (1968). The structure of inference. New York: Wiley. ISBN 0-471-27548-4. OCLC 440926.
↑ Fraser, D. A. S. (1979). Inference and linear models. London: McGraw-Hill. ISBN 0-07-021910-9. OCLC 3559629.
↑ Taraldsen, Gunnar; Lindqvist, Bo Henry (2013-02-01). "Fiducial theory and optimal inference". The Annals of Statistics. 41 (1). arXiv: 1301.1717 . doi:10.1214/13-AOS1083. ISSN 0090-5364. S2CID 88520957.
↑ De Finetti, Bruno (1937). "La Prévision: ses lois logiques, ses sources subjectives". Annales de l'Institut Henri Poincaré. 7 (1): 1–68. ISSN 0365-320X. Translated in De Finetti, Bruno (1992). "Foresight: Its Logical Laws, Its Subjective Sources". Breakthroughs in Statistics. Springer Series in Statistics. pp. 134–174. doi:10.1007/978-1-4612-0919-5_10. ISBN 978-0-387-94037-3.
↑ Geisser, Seymour (1993) Predictive Inference: An Introduction , CRC Press. ISBN 0-412-03471-9

Sources

Bandyopadhyay, P. S.; Forster, M. R., eds. (2011), Philosophy of Statistics, Elsevier .
Bickel, Peter J.; Doksum, Kjell A. (2001). Mathematical statistics: Basic and selected topics. Vol. 1 (Second (updated printing 2007) ed.). Prentice Hall. ISBN 978-0-13-850363-5. MR 0443141.
Cox, D. R. (2006). Principles of Statistical Inference, Cambridge University Press. ISBN 0-521-68567-2.
Fisher, R. A. (1955), "Statistical methods and scientific induction", Journal of the Royal Statistical Society, Series B, 17, 69–78. (criticism of statistical theories of Jerzy Neyman and Abraham Wald)
Freedman, D. A. (2009). Statistical Models: Theory and practice (revised ed.). Cambridge University Press. pp. xiv+442 pp. ISBN 978-0-521-74385-3. MR 2489600.
Freedman, D. A. (2010). Statistical Models and Causal Inferences: A Dialogue with the Social Sciences (Edited by David Collier, Jasjeet Sekhon, and Philip B. Stark), Cambridge University Press.
Hampel, Frank R. (February 2003). "The proper fiducial argument". Seminar für Statistik, Eidgenössische Technische Hochschule (ETH). 114. doi:10.3929/ethz-a-004526011.
Hansen, Mark H.; Yu, Bin (June 2001). "Model Selection and the Principle of Minimum Description Length: Review paper". Journal of the American Statistical Association . 96 (454): 746–774. CiteSeerX 10.1.1.43.6581 . doi:10.1198/016214501753168398. JSTOR 2670311. MR 1939352. S2CID 14460386. Archived from the original on 2004-11-16.
Hinkelmann, Klaus; Kempthorne, Oscar (2008). Introduction to Experimental Design (Second ed.). Wiley. ISBN 978-0-471-72756-9.
Kolmogorov, Andrei N. (1963). "On tables of random numbers". Sankhyā Ser. A. 25: 369–375. MR 0178484. Reprinted as Kolmogorov, Andrei N. (1998). "On tables of random numbers". Theoretical Computer Science . 207 (2): 387–395. doi: 10.1016/S0304-3975(98)00075-9 . MR 1643414.
Konishi S., Kitagawa G. (2008), Information Criteria and Statistical Modeling, Springer.
Kruskal, William (December 1988). "Miracles and statistics: the casual assumption of independence (ASA Presidential Address)". Journal of the American Statistical Association . 83 (404): 929–940. doi: 10.2307/2290117 . JSTOR 2290117.
Le Cam, Lucian. (1986) Asymptotic Methods of Statistical Decision Theory, Springer. ISBN 0-387-96307-3
Moore, D. S.; McCabe, G. P.; Craig, B. A. (2015), Introduction to the Practice of Statistics, Eighth Edition, Macmillan.
Neyman, Jerzy (1956). "Note on an article by Sir Ronald Fisher". Journal of the Royal Statistical Society, Series B . 18 (2): 288–294. doi:10.1111/j.2517-6161.1956.tb00236.x. JSTOR 2983716. (reply to Fisher 1955)
Peirce, C. S. (1877–1878), "Illustrations of the logic of science" (series), Popular Science Monthly , vols. 12–13. Relevant individual papers:
- (1878 March), "The Doctrine of Chances", Popular Science Monthly, v. 12, March issue, pp. 604–615. Internet Archive Eprint.
- (1878 April), "The Probability of Induction", Popular Science Monthly, v. 12, pp. 705–718. Internet Archive Eprint.
- (1878 June), "The Order of Nature", Popular Science Monthly, v. 13, pp. 203–217.Internet Archive Eprint.
- (1878 August), "Deduction, Induction, and Hypothesis", Popular Science Monthly, v. 13, pp. 470–482. Internet Archive Eprint.
Peirce, C. S. (1883), "A Theory of probable inference", Studies in Logic, pp. 126-181, Little, Brown, and Company. (Reprinted 1983, John Benjamins Publishing Company, ISBN 90-272-3271-7)
Freedman, D.A; Pisani, R.; Purves, R.A. (1978). Statistics . New York: W. W. Norton & Company.
Pfanzagl, Johann; with the assistance of R. Hamböker (1994). Parametric Statistical Theory. Berlin: Walter de Gruyter. ISBN 978-3-11-013863-4. MR 1291393.
Rissanen, Jorma (1989). Stochastic Complexity in Statistical Inquiry. Series in Computer Science. Vol. 15. Singapore: World Scientific. ISBN 978-9971-5-0859-3. MR 1082556.
Soofi, Ehsan S. (December 2000). "Principal information-theoretic approaches (Vignettes for the Year 2000: Theory and Methods, ed. by George Casella)". Journal of the American Statistical Association . 95 (452): 1349–1353. doi:10.1080/01621459.2000.10474346. JSTOR 2669786. MR 1825292. S2CID 120143121.
Traub, Joseph F.; Wasilkowski, G. W.; Wozniakowski, H. (1988). Information-Based Complexity. Academic Press. ISBN 978-0-12-697545-1.
Zabell, S. L. (Aug 1992). "R. A. Fisher and Fiducial Argument". Statistical Science. 7 (3): 369–387. doi: 10.1214/ss/1177011233 . JSTOR 2246073.

External links

Statistical Inference – lecture on the MIT OpenCourseWare platform
Statistical Inference – lecture by the National Programme on Technology Enhanced Learning
An online, Bayesian (MCMC) demo/calculator is available at causaScientia

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[7] According to Peirce, acceptance means that inquiry on this question ceases for the time being. In science, all scientific theories are revisable.

[Oxford-1] Upton, G., Cook, I. (2008) Oxford Dictionary of Statistics, OUP. ISBN 978-0-19-954145-4.

[2] "TensorFlow Lite inference". The term inference refers to the process of executing a TensorFlow Lite model on-device in order to make predictions based on input data.

[3] Johnson, Richard (12 March 2016). "Statistical Inference". Encyclopedia of Mathematics. Springer: The European Mathematical Society. Retrieved 26 October 2022.

[4] Konishi & Kitagawa (2008), p. 75.

[5] Cox (2006), p. 197.

[6] "Statistical inference - Encyclopedia of Mathematics". www.encyclopediaofmath.org. Retrieved 2019-01-23.

[Cox20062-8] 1 2 Cox (2006) page 2

[9] Evans, Michael; et al. (2004). Probability and Statistics: The Science of Uncertainty. Freeman and Company. p. 267. ISBN 9780716747420.

[10] van der Vaart, A.W. (1998) Asymptotic Statistics Cambridge University Press. ISBN 0-521-78450-6 (page 341)

[11] Kruskal 1988

[12] Freedman, D.A. (2008) "Survival analysis: An Epidemiological hazard?". The American Statistician (2008) 62: 110-119. (Reprinted as Chapter 11 (pages 169–192) of Freedman (2010)).

[13] Berk, R. (2003) Regression Analysis: A Constructive Critique (Advanced Quantitative Techniques in the Social Sciences) (v. 11) Sage Publications. ISBN 0-7619-2904-5

[Brewer2-14] 1 2 Brewer, Ken (2002). Combined Survey Sampling Inference: Weighing of Basu's Elephants. Hodder Arnold. p. 6. ISBN 978-0340692295.

[JHJ2-15] 1 2 Jörgen Hoffman-Jörgensen's Probability With a View Towards Statistics, Volume I. Page 399 ^{[ full citation needed ]}

[16] Le Cam (1986) ^{[ page needed ]}

[17] Erik Torgerson (1991) Comparison of Statistical Experiments, volume 36 of Encyclopedia of Mathematics. Cambridge University Press. ^{[ full citation needed ]}

[18] Liese, Friedrich & Miescke, Klaus-J. (2008). Statistical Decision Theory: Estimation, Testing, and Selection. Springer. ISBN 978-0-387-73193-3.

[19] Kolmogorov (1963, p.369): "The frequency concept, based on the notion of limiting frequency as the number of trials increases to infinity, does not contribute anything to substantiate the applicability of the results of probability theory to real practical problems where we have always to deal with a finite number of trials".

[20] "Indeed, limit theorems 'as $n$ tends to infinity' are logically devoid of content about what happens at any particular $n$ . All they can do is suggest certain approaches whose performance must then be checked on the case at hand." — Le Cam (1986) (page xiv)

[21] Pfanzagl (1994): "The crucial drawback of asymptotic theory: What we expect from asymptotic theory are results which hold approximately . . . . What asymptotic theory has to offer are limit theorems."(page ix) "What counts for applications are approximations, not limits." (page 188)

[22] Pfanzagl (1994) : "By taking a limit theorem as being approximately true for large sample sizes, we commit an error the size of which is unknown. [. . .] Realistic information about the remaining errors may be obtained by simulations." (page ix)

[23] Neyman, J.(1934) "On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection", Journal of the Royal Statistical Society , 97 (4), 557–625 JSTOR 2342192

[Hinkelmann_and_Kempthorne2-24] 1 2 Hinkelmann and Kempthorne(2008) ^{[ page needed ]}

[25] ASA Guidelines for the first course in statistics for non-statisticians. (available at the ASA website)

[26] David A. Freedman et alia's Statistics.

[27] Moore et al. (2015).

[28] Gelman A. et al. (2013). Bayesian Data Analysis (Chapman & Hall).

[29] Peirce (1877-1878)

[30] Peirce (1883)

[FOOTNOTEFreedmanPisaniPurves1978-31] Freedman, Pisani & Purves 1978.

[32] David A. Freedman Statistical Models.

[33] Rao, C.R. (1997) Statistics and Truth: Putting Chance to Work, World Scientific. ISBN 981-02-3111-3

[34] Peirce; Freedman; Moore et al. (2015).^{[ citation needed ]}

[35] Box, G.E.P. and Friends (2006) Improving Almost Anything: Ideas and Essays, Revised Edition, Wiley. ISBN 978-0-471-72755-2

[36] Cox (2006), p. 196.

[37] ASA Guidelines for the first course in statistics for non-statisticians. (available at the ASA website)
David A. Freedman et alias Statistics.
Moore et al. (2015).

[mwAwc] David A. Freedman et alias Statistics.

[mwAwk] Moore et al. (2015).

[38] Neyman, Jerzy. 1923 [1990]. "On the Application of Probability Theory to AgriculturalExperiments. Essay on Principles. Section 9." Statistical Science 5 (4): 465–472. Trans. Dorota M. Dabrowska and Terence P. Speed.

[39] Hinkelmann & Kempthorne (2008) ^{[ page needed ]}

[Dinov_Palanimalai_Khare_Christou_20182-40] Dinov, Ivo; Palanimalai, Selvam; Khare, Ashwini; Christou, Nicolas (2018). "Randomization-based statistical inference: A resampling and simulation infrastructure". Teaching Statistics. 40 (2): 64–73. doi:10.1111/test.12156. PMC 6155997 . PMID 30270947.

[41] Hinkelmann and Kempthorne (2008) Chapter 6.

[Dinov_Palanimalai_Khare_Christou_2018-42] Dinov, Ivo; Palanimalai, Selvam; Khare, Ashwini; Christou, Nicolas (2018). "Randomization-based statistical inference: A resampling and simulation infrastructure". Teaching Statistics. 40 (2): 64–73. doi:10.1111/test.12156. PMC 6155997 . PMID 30270947.

[Tang_model-based_Model-Free_2019-43] Tang, Ming; Gao, Chao; Goutman, Stephen; Kalinin, Alexandr; Mukherjee, Bhramar; Guan, Yuanfang; Dinov, Ivo (2019). "Model-Based and Model-Free Techniques for Amyotrophic Lateral Sclerosis Diagnostic Prediction and Patient Clustering". Neuroinformatics. 17 (3): 407–421. doi:10.1007/s12021-018-9406-9. PMC 6527505 . PMID 30460455.

[Politis_Model-Free_Inference_2019-44] Politis, D.N. (2019). "Model-free inference in statistics: how and why". IMS Bulletin. 48.

[45] Bandyopadhyay & Forster (2011). See the book's Introduction (p.3) and "Section III: Four Paradigms of Statistics".

[46] Neyman, J. (1937). "Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability". Philosophical Transactions of the Royal Society of London A. 236 (767): 333–380. Bibcode:1937RSPTA.236..333N. doi: 10.1098/rsta.1937.0005 . JSTOR 91337.

[47] Preface to Pfanzagl.

[48] Little, Roderick J. (2006). "Calibrated Bayes: A Bayes/Frequentist Roadmap". The American Statistician. 60 (3): 213–223. doi:10.1198/000313006X117837. ISSN 0003-1305. JSTOR 27643780. S2CID 53505632.

[49] Lee, Se Yoon (2021). "Gibbs sampler and coordinate ascent variational inference: A set-theoretical review". Communications in Statistics - Theory and Methods. 51 (6): 1549–1568. arXiv: 2008.01006 . doi:10.1080/03610926.2021.1921214. S2CID 220935477.

[Soofi_2000_1349–1353-50] Soofi (2000)

[HY-51] 1 2 Hansen & Yu (2001)

[HY747-52] 1 2 Hansen and Yu (2001), page 747.

[JR-53] 1 2 Rissanen (1989), page 84

[54] Joseph F. Traub, G. W. Wasilkowski, and H. Wozniakowski. (1988) ^{[ page needed ]}

[55] Neyman (1956)

[56] Zabell (1992)

[57] Cox (2006) page 66

[FOOTNOTEHampel2003-58] Hampel 2003.

[59] Davison, page 12. ^{[ full citation needed ]}

[60] Barnard, G.A. (1995) "Pivotal Models and the Fiducial Argument", International Statistical Review, 63 (3), 309–323. JSTOR 1403482

[61] Fraser, D. A. S. (1968). The structure of inference. New York: Wiley. ISBN 0-471-27548-4. OCLC 440926.

[62] Fraser, D. A. S. (1979). Inference and linear models. London: McGraw-Hill. ISBN 0-07-021910-9. OCLC 3559629.

[63] Taraldsen, Gunnar; Lindqvist, Bo Henry (2013-02-01). "Fiducial theory and optimal inference". The Annals of Statistics. 41 (1). arXiv: 1301.1717 . doi:10.1214/13-AOS1083. ISSN 0090-5364. S2CID 88520957.

[64] De Finetti, Bruno (1937). "La Prévision: ses lois logiques, ses sources subjectives". Annales de l'Institut Henri Poincaré. 7 (1): 1–68. ISSN 0365-320X. Translated in De Finetti, Bruno (1992). "Foresight: Its Logical Laws, Its Subjective Sources". Breakthroughs in Statistics. Springer Series in Statistics. pp. 134–174. doi:10.1007/978-1-4612-0919-5_10. ISBN 978-0-387-94037-3.

[geisser-65] Geisser, Seymour (1993) Predictive Inference: An Introduction , CRC Press. ISBN 0-412-03471-9

[1]

[2]

[3]

[4]

[5]

[6]

[note 1]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[52]

[53]

[54]

[55]

[56]

[57]

[58]

[59]

[60]

[61]

[62]

[63]

[64]