Search Results

Divide and conquer algorithm is a common strategy applied in big data. Model averaging has the natural divide-and-conquer feature, but its theory has not been developed in big data scenarios. The goal of this paper is to fill this gap. We propose two ...

We compare the performance of six model average predictors--Mallows' model averaging, stacking, Bayes model averaging, bagging, random forests, and boosting--to the components used to form them. In all six cases we identify conditions under which the ...

In this paper we adopt the familiar sparse, high-dimensional linear regression model and focus on the important but often overlooked task of prediction. In particular, we consider a new empirical Bayes framework that incorporates data in the prior in two ...

Model fusion methods, or more generally ensemble methods, are a useful tool for prediction. Combining predictions from a set of models smooths out biases and reduces variances of predictions from individual models, and hence, the combined predictions ...

This work characterizes the benefits of averaging techniques widely used in conjunction with stochastic gradient descent (SGD). In particular, this work presents a sharp analysis of: (1) mini-batching, a method of averaging many samples of a stochastic ...

This paper presents a study of using deep bidirectional long short-term memory (DBLSTM) recurrent neural network as acoustic model for DBLSTM-HMM based large vocabulary continuous speech recognition (LVCSR), where a context-sensitive-chunk (CSC) back-...

A wide variety of audio source separation techniques exist and can already tackle many challenging industrial issues. However, in contrast with other application domains, fusion principles were rarely investigated in audio source separation despite ...

Focusing on credit risk modelling, this paper introduces a novel approach for ensemble modelling based on a normative linear pooling. Models are first classified as dominant and competitive, and the pooling is run using the competitive models only. ...

Microarray technology allows for the collection of multiple replicates of gene expression time course data for hundreds of genes at a handful of time points. Developing hypotheses about a gene transcriptional network, based on time course gene ...

In this paper, a model averaging approach is developed for the linear factor regression model in light of smoothed focused information criterion. With respect to factors, a frequentist model averaging estimation of the regression parameter is proposed ...

Naive Bayes (NB) is a simple Bayesian classifier that assumes the conditional independence and augmented NB (ANB) models are extensions of NB by relaxing the independence assumption. The averaged one-dependence estimators (AODE) is a classifier that ...

Classification is an important data analysis tool that uses a model built from historical data to predict class labels for new observations. More and more applications are featuring data streams, rather than finite stored data sets, which are a ...

This poster introduces a novel approach to information retrieval that uses statistical model averaging to improve latent semantic indexing (LSI). Instead of choosing a single dimensionality $k$ for LSI , we propose using several models of differing ...

The maximum likelihood linear regression (MLLR) technique is widely used in speaker adaptation due to its effectiveness and computational advantages. When the adaptation data are sparse, MLLR performance degrades because of unreliable parameter ...

We develop a statistical method for estimating the spectrum from a data set that consists of several signals, all of which are realizations of a common random process. We first find estimates of the common spectrum using each signal; then we construct M ...

PAC-Bayesian learning methods combine the informative priors of Bayesian methods with distribution-free PAC guarantees. Stochastic model selection predicts a class label by stochastically sampling a classifier according to a “posterior distribution” on ...

Given a set of possible models for variables X and a set of possible parameters for each model, the Bayesian “estimate” of the probability distribution for X given observed data is obtained by averaging over the possible models and their parameters. An ...

When performing regression or classification, we are interested in the conditional probability distribution for an outcome or class variable Y given a set of explanatory or input variables X. We consider Bayesian models for this task. In particular, we ...