Information geometry is an interdisciplinary field that applies the techniques of differential geometry to study probability theory and statistics. [1] It studies statistical manifolds, which are Riemannian manifolds whose points correspond to probability distributions.
 | This article may need to be rewritten to comply with Wikipedia's quality standards, as This should be edited to include some statistical background..(April 2019) |
Historically, information geometry can be traced back to the work of C. R. Rao, who was the first to treat the Fisher matrix as a Riemannian metric. [2] [3] The modern theory is largely due to Shun'ichi Amari, whose work has been greatly influential on the development of the field. [4]
Classically, information geometry considered a parametrized statistical model as a Riemannian manifold. For such models, there is a natural choice of Riemannian metric, known as the Fisher information metric. In the special case that the statistical model is an exponential family, it is possible to induce the statistical manifold with a Hessian metric (i.e a Riemannian metric given by the potential of a convex function). In this case, the manifold naturally inherits two flat affine connections, as well as a canonical Bregman divergence. Historically, much of the work was devoted to studying the associated geometry of these examples. In the modern setting, information geometry applies to a much wider context, including non-exponential families, nonparametric statistics, and even abstract statistical manifolds not induced from a known statistical model. The results combine techniques from information theory, affine differential geometry, convex analysis and many other fields. One of the most perspective information geometry approaches find applications in machine learning. For example, the developing of information-geometric optimization methods (mirror descent [5] and natural gradient descent [6] ).
The standard references in the field are Shun’ichi Amari and Hiroshi Nagaoka's book, Methods of Information Geometry, [7] and the more recent book by Nihat Ay and others. [8] A gentle introduction is given in the survey by Frank Nielsen. [9] In 2018, the journal Information Geometry was released, which is devoted to the field.
| | This section may need to be rewritten to comply with Wikipedia's quality standards, as Does this list make sense in a general article?.(May 2013) |
The history of information geometry is associated with the discoveries of at least the following people, and many others.
| | This section may need to be rewritten to comply with Wikipedia's quality standards, as References should be provided for these..(April 2019) |
As an interdisciplinary field, information geometry has been used in various applications.
Here an incomplete list:
In mathematics, differential topology is the field dealing with the topological properties and smooth properties of smooth manifolds. In this sense differential topology is distinct from the closely related field of differential geometry, which concerns the geometric properties of smooth manifolds, including notions of size, distance, and rigid shape. By comparison differential topology is concerned with coarser properties, such as the number of holes in a manifold, its homotopy type, or the structure of its diffeomorphism group. Because many of these coarser properties may be captured algebraically, differential topology has strong links to algebraic topology.
Differential geometry is a mathematical discipline that studies the geometry of smooth shapes and smooth spaces, otherwise known as smooth manifolds. It uses the techniques of differential calculus, integral calculus, linear algebra and multilinear algebra. The field has its origins in the study of spherical geometry as far back as antiquity. It also relates to astronomy, the geodesy of the Earth, and later the study of hyperbolic geometry by Lobachevsky. The simplest examples of smooth spaces are the plane and space curves and surfaces in the three-dimensional Euclidean space, and the study of these shapes formed the basis for development of modern differential geometry during the 18th and 19th centuries.
Riemannian geometry is the branch of differential geometry that studies Riemannian manifolds, defined as smooth manifolds with a Riemannian metric. This gives, in particular, local notions of angle, length of curves, surface area and volume. From those, some other global quantities can be derived by integrating local contributions.
Shing-Tung Yau is a Chinese-American mathematician. He is the director of the Yau Mathematical Sciences Center at Tsinghua University and professor emeritus at Harvard University. Until 2022, Yau was the William Caspar Graustein Professor of Mathematics at Harvard, at which point he moved to Tsinghua.
In information geometry, the Fisher information metric is a particular Riemannian metric which can be defined on a smooth statistical manifold, i.e., a smooth manifold whose points are probability measures defined on a common probability space. It can be used to calculate the informational difference between measurements.
In mathematical statistics, the Fisher information is a way of measuring the amount of information that an observable random variable X carries about an unknown parameter θ of a distribution that models X. Formally, it is the variance of the score, or the expected value of the observed information.
In the mathematical field of differential geometry, Ricci-flatness is a condition on the curvature of a Riemannian manifold. Ricci-flat manifolds are a special kind of Einstein manifold. In theoretical physics, Ricci-flat Lorentzian manifolds are of fundamental interest, as they are the solutions of Einstein's field equations in a vacuum with vanishing cosmological constant.
Richard Streit Hamilton was an American mathematician who served as the Davies Professor of Mathematics at Columbia University.
In mathematical statistics, the Kullback–Leibler (KL) divergence, denoted , is a type of statistical distance: a measure of how one reference probability distribution P is different from a second probability distribution Q. Mathematically, it is defined as
A stochastic differential equation (SDE) is a differential equation in which one or more of the terms is a stochastic process, resulting in a solution which is also a stochastic process. SDEs have many applications throughout pure mathematics and are used to model various behaviours of stochastic models such as stock prices, random growth models or physical systems that are subjected to thermal fluctuations.
Ruppeiner geometry is thermodynamic geometry using the language of Riemannian geometry to study thermodynamics. George Ruppeiner proposed it in 1979. He claimed that thermodynamic systems can be represented by Riemannian geometry, and that statistical properties can be derived from the model.
Jean-Michel Bismut is a French mathematician who has been a professor at the Université Paris-Sud since 1981. His mathematical career covers two apparently different branches of mathematics: probability theory and differential geometry. Ideas from probability play an important role in his works on geometry.
Damiano Brigo is a mathematician known for research in mathematical finance, filtering theory, stochastic analysis with differential geometry, probability theory and statistics, authoring more than 130 research publications and three monographs. From 2012 he serves as full professor with a chair in mathematical finance at the Department of Mathematics of Imperial College London, where he headed the Mathematical Finance group in 2012–2019. He is also a well known quantitative finance researcher, manager and advisor in the industry. His research has been cited and published also in mainstream industry publications, including Risk Magazine, where he has been the most cited author in the twenty years 1998–2017. He is often requested as a plenary or invited speaker both at academic and industry international events. Brigo's research has also been used in court as support for legal proceedings.
In information geometry, a divergence is a kind of statistical distance: a binary function which establishes the separation from one probability distribution to another on a statistical manifold.
Mathematics is a broad subject that is commonly divided in many areas that may be defined by their objects of study, by the used methods, or by both. For example, analytic number theory is a subarea of number theory devoted to the use of methods of analysis for the study of natural numbers.
In mathematics, a statistical manifold is a Riemannian manifold, each of whose points is a probability distribution. Statistical manifolds provide a setting for the field of information geometry. The Fisher information metric provides a metric on these manifolds. Following this definition, the log-likelihood function is a differentiable map and the score is an inclusion.
Jürgen Jost is a German mathematician specializing in geometry. He has been a director of the Max Planck Institute for Mathematics in the Sciences in Leipzig since 1996.
In information geometry, Chentsov's theorem states that the Fisher information metric is, up to rescaling, the unique Riemannian metric on a statistical manifold that is invariant under sufficient statistics.
Projection filters are a set of algorithms based on stochastic analysis and information geometry, or the differential geometric approach to statistics, used to find approximate solutions for filtering problems for nonlinear state-space systems. The filtering problem consists of estimating the unobserved signal of a random dynamical system from partial noisy observations of the signal. The objective is computing the probability distribution of the signal conditional on the history of the noise-perturbed observations. This distribution allows for calculations of all statistics of the signal given the history of observations. If this distribution has a density, the density satisfies specific stochastic partial differential equations (SPDEs) called Kushner-Stratonovich equation, or Zakai equation. It is known that the nonlinear filter density evolves in an infinite dimensional function space.
Differentiable computing | |||||||
|---|---|---|---|---|---|---|---|
| General | |||||||
| Concepts | |||||||
| Applications | |||||||
| Hardware | |||||||
| Software libraries | |||||||
| Implementations |
| ||||||
| People | |||||||
| Organizations | |||||||
| Architectures |
| ||||||
| |||||||
