Search Results (25)

Population genetics describes evolutionary processes, focusing on the variation within and between species and the forces shaping this diversity. Evolution reflects information accumulated in genomes, enhancing organisms’ adaptation to their environment. In this paper, I propose a model that begins with the distribution of mating based on mutual fitness and progresses to viable adult genotype distribution. At each stage, the changes result in different measures of information. The evolutionary dynamics at each stage of the model correspond to certain aspects of interest, such as the type of mating, the distribution of genotypes in regard to mating, and the distribution of genotypes and haplotypes in the next generation. Changes to these distributions are caused by variations in fitness and result in Jeffrey’s divergence values other than zero. As an example, a model of hybrid sterility is developed of a biallelic locus, comparing the information indices associated with each stage of the evolutionary process. In conclusion, the informational perspective seems to facilitate the connection between cause and effect and allows the development of statistical tests to perform hypothesis testing against zero-information null models (random mating, no selection, etc.). The informational perspective could contribute to clarify, deepen, and expand the mathematical foundations of evolutionary theory. Full article

In this study, we consider the problem of self-supervised learning (SSL) utilizing the 1-Wasserstein distance on a tree structure (a.k.a., Tree-Wasserstein distance (TWD)), where TWD is defined as the L1 distance between two tree-embedded vectors. In SSL methods, the cosine similarity is often utilized as an objective function; however, it has not been well studied when utilizing the Wasserstein distance. Training the Wasserstein distance is numerically challenging. Thus, this study empirically investigates a strategy for optimizing the SSL with the Wasserstein distance and finds a stable training procedure. More specifically, we evaluate the combination of two types of TWD (total variation and ClusterTree) and several probability models, including the softmax function, the ArcFace probability model, and simplicial embedding. We propose a simple yet effective Jeffrey divergence-based regularization method to stabilize optimization. Through empirical experiments on STL10, CIFAR10, CIFAR100, and SVHN, we find that a simple combination of the softmax function and TWD can obtain significantly lower results than the standard SimCLR. Moreover, a simple combination of TWD and SimSiam fails to train the model. We find that the model performance depends on the combination of TWD and probability model, and that the Jeffrey divergence regularization helps in model training. Finally, we show that the appropriate combination of the TWD and probability model outperforms cosine similarity-based representation learning. Full article

We investigate the asymptotic properties of the plug-in estimator for the Jeffreys divergence, the symmetric variant of the Kullback–Leibler (KL) divergence. This study focuses specifically on the divergence between discrete distributions. Traditionally, estimators rely on two independent samples corresponding to two distinct conditions. However, we propose a one-sample estimator where the condition results from a random event. We establish the estimator’s asymptotic unbiasedness (law of large numbers) and asymptotic normality (central limit theorem). Although the results are expected, the proofs require additional technical work due to the randomness of the conditions. Full article

In this paper, we use the generalized version of convex functions, known as strongly convex functions, to derive improvements to the Jensen–Mercer inequality. We achieve these improvements through the newly discovered characterizations of strongly convex functions, along with some previously known results about strongly convex functions. We are also focused on important applications of the derived results in information theory, deducing estimates for χ-divergence, Kullback–Leibler divergence, Hellinger distance, Bhattacharya distance, Jeffreys distance, and Jensen–Shannon divergence. Additionally, we prove some applications to Mercer-type power means at the end. Full article

A measurement performed on a quantum system is an act of gaining information about its state. However, in the foundations of quantum theory, the concept of information is multiply defined, particularly in the area of quantum reconstruction, and its conceptual foundations remain surprisingly under-explored. In this paper, we investigate the gain of information in quantum measurements from an operational viewpoint in the special case of a two-outcome probabilistic source. We show that the continuous extension of the Shannon entropy naturally admits two distinct measures of information gain, differential information gain and relative information gain, and that these have radically different characteristics. In particular, while differential information gain can increase or decrease as additional data are acquired, relative information gain consistently grows and, moreover, exhibits asymptotic indifference to the data or choice of Bayesian prior. In order to make a principled choice between these measures, we articulate a Principle of Information Increase, which incorporates a proposal due to Summhammer that more data from measurements leads to more knowledge about the system, and also takes into consideration black swan events. This principle favours differential information gain as the more relevant metric and guides the selection of priors for these information measures. Finally, we show that, of the symmetric beta distribution priors, the Jeffreys binomial prior is the prior that ensures maximal robustness of information gain for the particular data sequence obtained in a run of experiments. Full article

In this paper, we present the derivation of Jeffreys divergence, generalized Fisher divergence, and the corresponding De Bruijn identities for space–time random field. First, we establish the connection between Jeffreys divergence and generalized Fisher information of a single space–time random field with respect to time and space variables. Furthermore, we obtain the Jeffreys divergence between two space–time random fields obtained by different parameters under the same Fokker–Planck equations. Then, the identities between the partial derivatives of the Jeffreys divergence with respect to space–time variables and the generalized Fisher divergence are found, also known as the De Bruijn identities. Later, at the end of the paper, we present three examples of the Fokker–Planck equations on space–time random fields, identify their density functions, and derive the Jeffreys divergence, generalized Fisher information, generalized Fisher divergence, and their corresponding De Bruijn identities. Full article

We present a simple method to approximate the Fisher–Rao distance between multivariate normal distributions based on discretizing curves joining normal distributions and approximating the Fisher–Rao distances between successive nearby normal distributions on the curves by the square roots of their Jeffreys divergences. We consider experimentally the linear interpolation curves in the ordinary, natural, and expectation parameterizations of the normal distributions, and compare these curves with a curve derived from the Calvo and Oller’s isometric embedding of the Fisher–Rao d-variate normal manifold into the cone of (d+1)×(d+1) symmetric positive–definite matrices. We report on our experiments and assess the quality of our approximation technique by comparing the numerical approximations with both lower and upper bounds. Finally, we present several information–geometric properties of Calvo and Oller’s isometric embedding. Full article

This paper aims at certain theoretical studies and additional computational analysis on symmetry and its lack in Kullback-Leibler and Jeffreys probabilistic divergences related to some engineering applications. As it is known, the Kullback-Leibler distance in between two different uncertainty sources exhibits a lack of symmetry, while the Jeffreys model represents its symmetrization. The basic probabilistic computational implementation has been delivered in the computer algebra system MAPLE 2019^®, whereas engineering illustrations have been prepared with the use of the Finite Element Method systems Autodesk ROBOT^® & ABAQUS^®. Determination of the first two probabilistic moments fundamental in the calculation of both relative entropies has been made (i) analytically, using a semi-analytical approach (based upon the series of the FEM experiments), and (ii) the iterative generalized stochastic perturbation technique, where some reference solutions have been delivered using (iii) Monte-Carlo simulation. Numerical analysis proves the fundamental role of computer algebra systems in probabilistic entropy determination and shows remarkable differences obtained with the two aforementioned relative entropy models, which, in some specific cases, may be neglected. As it is demonstrated in this work, a lack of symmetry in probabilistic divergence may have a decisive role in engineering reliability, where extreme and admissible responses cannot be simply replaced with each other in any case. Full article

In this paper, we introduce new divergences called Jensen–Sharma–Mittal and Jeffreys–Sharma–Mittal in relation to convex functions. Some theorems, which give the lower and upper bounds for two new introduced divergences, are provided. The obtained results imply some new inequalities corresponding to known divergences. Some examples, which show that these are the generalizations of Rényi, Tsallis, and Kullback–Leibler types of divergences, are provided in order to show a few applications of new divergences. Full article

The present paper investigates the update of an empirical probability distribution with the results of a new set of observations. The update reproduces the new observations and interpolates using prior information. The optimal update is obtained by minimizing either the Hellinger distance or the quadratic Bregman divergence. The results obtained by the two methods differ. Updates with information about conditional probabilities are considered as well. Full article

The Jeffreys divergence is a renown arithmetic symmetrization of the oriented Kullback–Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various techniques with advantages and disadvantages have been proposed in the literature to either estimate, approximate, or lower and upper bound this divergence. In this paper, we propose a simple yet fast heuristic to approximate the Jeffreys divergence between two univariate Gaussian mixtures with arbitrary number of components. Our heuristic relies on converting the mixtures into pairs of dually parameterized probability densities belonging to an exponential-polynomial family. To measure with a closed-form formula the goodness of fit between a Gaussian mixture and an exponential-polynomial density approximating it, we generalize the Hyvärinen divergence to α-Hyvärinen divergences. In particular, the 2-Hyvärinen divergence allows us to perform model selection by choosing the order of the exponential-polynomial densities used to approximate the mixtures. We experimentally demonstrate that our heuristic to approximate the Jeffreys divergence between mixtures improves over the computational time of stochastic Monte Carlo estimations by several orders of magnitude while approximating the Jeffreys divergence reasonably well, especially when the mixtures have a very small number of modes. Full article

This article proposes the use of the Bayesian reference analysis to estimate the parameters of the generalized normal linear regression model. It is shown that the reference prior led to a proper posterior distribution, while the Jeffreys prior returned an improper one. The inferential purposes were obtained via Markov Chain Monte Carlo (MCMC). Furthermore, diagnostic techniques based on the Kullback–Leibler divergence were used. The proposed method was illustrated using artificial data and real data on the height and diameter of Eucalyptus clones from Brazil. Full article

We give bounds on the difference between the weighted arithmetic mean and the weighted geometric mean. These imply refined Young inequalities and the reverses of the Young inequality. We also studied some properties on the difference between the weighted arithmetic mean and the weighted geometric mean. Applying the newly obtained inequalities, we show some results on the Tsallis divergence, the Rényi divergence, the Jeffreys–Tsallis divergence and the Jensen–Shannon–Tsallis divergence. Full article

In this work, we first consider the discrete version of Fisher information measure and then propose Jensen–Fisher information, to develop some associated results. Next, we consider Fisher information and Bayes–Fisher information measures for mixing parameter vector of a finite mixture probability mass function and establish some results. We provide some connections between these measures with some known informational measures such as chi-square divergence, Shannon entropy, Kullback–Leibler, Jeffreys and Jensen–Shannon divergences. Full article

Home advantage in sports is important for coaches, players, fans, and commentators and has a key role in sports prediction models. This paper builds on results of recent research that—instead of points gained—used goals scored and goals conceded to describe home advantage. This offers more detailed look at this phenomenon. Presented description understands a home advantage in leagues as a random variable that can be described by a trinomial distribution. The paper uses this description to offer new ways of home advantage comparison—based on the Jeffrey divergence and the test for homogeneity—in different leagues. Next, a heuristic procedure—based on distances between probability descriptions of home advantage in leagues—is developed for identification of leagues with similar home advantage. Publicly available data are used for demonstration of presented procedures in 19 European football leagues between the 2007/2008 and 2016/2017 seasons, and for individual teams of one league in one season. Overall, the highest home advantage rate was identified in the highest Greek football league, and the lowest was identified in the fourth level English football league. Full article