research-article

Particle Algorithms for Optimization on Binary Spaces

Author:

Christian SchäferAuthors Info & Claims

ACM Transactions on Modeling and Computer Simulation (TOMACS), Volume 23, Issue 1

Article No.: 8, Pages 1 - 25

https://doi.org/10.1145/2414416.2414424

Published: 01 January 2013 Publication History

Abstract

We discuss a unified approach to stochastic optimization of pseudo-Boolean objective functions based on particle methods, including the cross-entropy method and simulated annealing as special cases. We point out the need for auxiliary sampling distributions, meaning parametric families on binary spaces, which are able to reproduce complex dependency structures, and illustrate their usefulness in our numerical experiments. We provide numerical evidence that particle-driven optimization algorithms based on parametric families yield superior results on strongly multimodal optimization problems while local search heuristics outperform them on easier problems.

Supplementary Material

PDF File (a8-schafer_appendix.pdf)

The proof is given in an electronic appendix, available online in the ACM Digital Library.

Download
89.73 KB

References

[1]

Alidaee, B., Kochenberger, G., and Wang, H. 2010. Theorems supporting r-flip search for pseudo-Boolean optimization. Int. J. Appl. Metaheur. Comput. (IJAMC) 1, 1, 93--109.

Digital Library

[2]

Amini, M., Alidaee, B., and Kochenberger, G. 1999. A scatter search approach to unconstrained quadratic binary programs. In New Ideas in Optimization. McGraw-Hill Ltd., UK, 317--330.

Digital Library

[3]

Beasley, J. 1990. OR-Library: Distributing test problems by electronic mail. J. Operat. Res. Soc., 1069--1072.

[4]

Beasley, J. 1998. Heuristic algorithms for the unconstrained binary quadratic programming problem. Tech. rep., Management School, Imperial College London.

[5]

Billionnet, A. and Sutter, A. 1994. Minimization of a quadratic pseudo-Boolean function. Europ. J. Oper. Res. 78, 1, 106--115.

[6]

Boros, E. and Hammer, P. 2002. Pseudo-Boolean optimization. Discr. Appl. Math. 123, 1--3, 155--225.

Digital Library

[7]

Boros, E., Hammer, P., and Tavares, G. 2007. Local search heuristics for quadratic unconstrained binary optimization (QUBO). J. Heuris. 13, 2, 99--132.

Digital Library

[8]

Carpenter, J., Clifford, P., and Fearnhead, P. 1999. Improved Particle Filter for nonlinear problems. IEE Proc. Radar, Sonar Nav. 146, 1, 2--7.

[9]

Chopin, N. 2002. A sequential particle filter method for static models. Biometrika 89, 3, 539.

[10]

Del Moral, P., Doucet, A., and Jasra, A. 2006. Sequential Monte Carlo samplers. J. Roy. Stat. Soc. Ser. B (Stat. Meth.) 68, 3, 411--436.

[11]

Del Moral, P., Doucet, A., and Jasra, A. 2012. An adaptive sequential Monte Carlo method for approximate Bayesian computation. Stat. Comput. to appear.

Digital Library

[12]

Emrich, L. and Piedmonte, M. 1991. A method for generating high-dimensional multivariate binary variates. Amer. Statist. 45, 302--304.

[13]

Fearnhead, P. and Clifford, P. 2003. Online inference for hidden Markov models via particle filters. J. Roy. Stat. Soc. Ser. B (Stat. Meth.) 65, 4, 887--899.

[14]

Garey, M. and Johnson, D. 1979. Computers and Intractability: A Guide to the Theory of NP-completeness. WH Freeman & Co.

Digital Library

[15]

Genz, A. and Bretz, F. 2009. Computation of Multivariate Normal and T Probabilities. Vol. 195. Springer.

Digital Library

[16]

Glover, F., Kochenberger, G., and Alidaee, B. 1998. Adaptive memory tabu search for binary quadratic programs. Manage. Sci. 44, 336--345.

Digital Library

[17]

Gordon, N. J., Salmond, D. J., and Smith, A. F. M. 1993. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. Radar, Sonar Navigation 140, 2, 107--113.

[18]

Gueye, S. and Michelon, P. 2009. A linearization framework for unconstrained quadratic (0-1) problems. Discr. Appl. Math. 157, 6, 1255--1266.

Digital Library

[19]

Hansen, P. and Meyer, C. 2009. Improved compact linearizations for the unconstrained quadratic 0-1 minimization problem. Discr. Appl. Math. 157, 6, 1267--1290.

Digital Library

[20]

Hasan, M., AlKhamis, T., and Ali, J. 2000. A comparison between simulated annealing, genetic algorithm and tabu search methods for the unconstrained quadratic Pseudo-Boolean function. Comput. Indust. Eng. 38, 3, 323--340.

Digital Library

[21]

Jasra, A., Stephens, D., Doucet, A., and Tsagaris, T. 2011. Inference for Lévy-Driven Stochastic Volatility Models via Adaptive Sequential Monte Carlo. Scand. J. Stat.

[22]

Joe, H. 1996. Families of m-variate distributions with given margins and m (m-1)/2 bivariate dependence parameters. Lecture Notes-Monograph Series 28, 120--141.

[23]

Johansen, A., Moral, P. D., and Doucet, A. 2006. Sequential Monte Carlo samplers for rare events. In Proceedings of the 6th International Workshop on Rare Event Simulation. 256--267.

[24]

Katayama, K. and Narihisa, H. 2001. Performance of Simulated Annealing-based heuristic for the unconstrained binary quadratic programming problem. Europ. J. Operat. Res. 134, 1, 103--119.

[25]

Kirkpatrick, S., Gelatt, C., and Vecchi, M. 1983. Optimization by simulated annealing. Science 220, 4598, 671.

[26]

Kitagawa, G. 1996. Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Computat. Graph. Stat. 5, 1, 1--25.

[27]

Kong, A., Liu, J. S., and Wong, W. H. 1994. Sequential imputation and Bayesian missing data problems. JASA 89, 278--288.

[28]

Kozlov, M., Tarasov, S., and Khachiyan, L. 1979. Polynomial solvability of convex quadratic programming. In Soviet Mathematics Doklady 20, 1108--1111.

[29]

Liu, J. and Chen, R. 1998. Sequential Monte Carlo methods for dynamic systems. JASA 93, 443, 1032--1044.

[30]

Merz, P. and Freisleben, B. 1999. Genetic algorithms for binary quadratic programming. In Proceedings of the Genetic and Evolutionary Computation Conference. Vol. 1. Citeseer, 417--424.

[31]

Merz, P. and Freisleben, B. 2002. Greedy and local search heuristics for unconstrained binary quadratic programming. J. Heur. 8, 2, 197--213.

Digital Library

[32]

Merz, P. and Katayama, K. 2004. Memetic algorithms for the unconstrained binary quadratic programming problem. BioSystems 78, 1--3, 99--118.

[33]

Nott, D. and Kohn, R. 2005. Adaptive sampling for Bayesian variable selection. Biometrika 92, 4, 747.

[34]

Palubeckis, G. 1995. A heuristic-based branch and bound algorithm for unconstrained quadratic zero-one programming. Computing 54, 4, 283--301.

[35]

Palubeckis, G. 2004. Multistart tabu search strategies for the unconstrained binary quadratic optimization problem. Ann. Oper. Res. 131, 1, 259--282.

[36]

Pardalos, P. 1991. Construction of test problems in quadratic bivalent programming. ACM Trans. Math. Softw. (TOMS) 17, 1, 74--87.

Digital Library

[37]

Pardalos, P. and Rodgers, G. 1990. Computational aspects of a branch and bound algorithm for quadratic zero-one programming. Computing 45, 2, 131--144.

Digital Library

[38]

Poljak, S. and Wolkowicz, H. 1995. Convex relaxations of (0, 1)-quadratic programming. Math. Oper. Res., 550--561.

Digital Library

[39]

Robert, C. and Casella, G. 2004. Monte Carlo Statistical Methods. Springer Verlag.

Digital Library

[40]

Rubinstein, R. Y. 1997. Optimization of computer simulation models with rare events. Europ. J. Oper. Res. 99, 89--112.

[41]

Rubinstein, R. Y. 1999. The Cross-Entropy Method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability 1, 127--190.

Digital Library

[42]

Rubinstein, R. Y. and Kroese, D. P. 2004. The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning. Springer-Verlag.

Digital Library

[43]

Schäfer, C. 2010. Parametric families on binary spaces. Tech. rep. arXiv:1008.0055.

[44]

Schäfer, C. 2012. On parametric families for sampling binary data with specified mean and correlation. Tech. rep. arXiv:1111.0576.

[45]

Schäfer, C. and Chopin, N. 2012. Sequential Monte Carlo on large binary sampling spaces. Stat. Comput. to appear.

Cited By

Mahmoudi ZDel Favero SJacob PChoudhary P(2021)Toward an Optimal Definition of Hypoglycemia with Continuous Glucose MonitoringComputer Methods and Programs in Biomedicine10.1016/j.cmpb.2021.106303(106303)Online publication date: Jul-2021
https://doi.org/10.1016/j.cmpb.2021.106303
Chopin NPapaspiliopoulos OChopin NPapaspiliopoulos O(2020)Beyond State-Space ModelsAn Introduction to Sequential Monte Carlo10.1007/978-3-030-47845-2_3(27-34)Online publication date: 2-Oct-2020
https://doi.org/10.1007/978-3-030-47845-2_3
Chopin NPapaspiliopoulos OChopin NPapaspiliopoulos O(2020)SMC SamplersAn Introduction to Sequential Monte Carlo10.1007/978-3-030-47845-2_17(329-355)Online publication date: 2-Oct-2020
https://doi.org/10.1007/978-3-030-47845-2_17
Show More Cited By

Index Terms

Particle Algorithms for Optimization on Binary Spaces

Recommendations

A fuzzy adaptive turbulent particle swarm optimisation

Particle Swarm Optimisation (PSO) algorithm is a stochastic search technique, which has exhibited good performance across a wide range of applications. However, very often for multimodal problems involving high dimensions, the algorithm tends to suffer ...
A Hybrid Cooperative Co-evolution Particle Swarm Optimizer for Function Optimization
AICI '10: Proceedings of the 2010 International Conference on Artificial Intelligence and Computational Intelligence - Volume 01

A novel cooperative co-evolution particle swarm optimizer advanced through bringing in quantum-behaved theory and simulated annealing(SA) method(HCPSO)is proposed in this paper, which makes full use advantages containing great global searching ability ...
Memetic binary particle swarm optimization for discrete optimization problems

In recent decades, many researchers have been interested in algorithms inspired by the observation of natural phenomena to solve optimization problems. Among them, meta-heuristic algorithms have been extensively applied in continuous (real) and discrete ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Computer Simulation

ACM Transactions on Modeling and Computer Simulation Volume 23, Issue 1

Special Issue on Monte Carlo Methods in Statistics

January 2013

207 pages

ISSN:1049-3301

EISSN:1558-1195

DOI:10.1145/2414416

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 January 2013

Accepted: 01 September 2012

Revised: 01 July 2012

Received: 01 October 2011

Published in TOMACS Volume 23, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
274
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)1

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mahmoudi ZDel Favero SJacob PChoudhary P(2021)Toward an Optimal Definition of Hypoglycemia with Continuous Glucose MonitoringComputer Methods and Programs in Biomedicine10.1016/j.cmpb.2021.106303(106303)Online publication date: Jul-2021
https://doi.org/10.1016/j.cmpb.2021.106303
Chopin NPapaspiliopoulos OChopin NPapaspiliopoulos O(2020)Beyond State-Space ModelsAn Introduction to Sequential Monte Carlo10.1007/978-3-030-47845-2_3(27-34)Online publication date: 2-Oct-2020
https://doi.org/10.1007/978-3-030-47845-2_3
Chopin NPapaspiliopoulos OChopin NPapaspiliopoulos O(2020)SMC SamplersAn Introduction to Sequential Monte Carlo10.1007/978-3-030-47845-2_17(329-355)Online publication date: 2-Oct-2020
https://doi.org/10.1007/978-3-030-47845-2_17
Baptista RMarzouk YWillcox KPeherstorfer B(2018)Optimal Approximations of Coupling in Multidisciplinary ModelsAIAA Journal10.2514/1.J05688856:6(2412-2428)Online publication date: Jun-2018
https://doi.org/10.2514/1.J056888
Baptista RMarzouk YWillcox KPeherstorfer B(2017)Optimal Approximations of Coupling in Multidisciplinary Models58th AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference10.2514/6.2017-1935Online publication date: 5-Jan-2017
https://doi.org/10.2514/6.2017-1935

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents