research-article

Bivariate estimation-of-distribution algorithms can find an exponential number of optima

Authors:

Benjamin Doerr,

Martin S. KrejcaAuthors Info & Claims

GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference

Pages 796 - 804

https://doi.org/10.1145/3377930.3390177

Published: 26 June 2020 Publication History

Abstract

Finding a large set of optima in a multimodal optimization landscape is a challenging task. Classical population-based evolutionary algorithms (EAs) typically converge only to a single solution. While this can be counteracted by applying niching strategies, the number of optima is nonetheless trivially bounded by the population size.

Estimation-of-distribution algorithms (EDAs) are an alternative, maintaining a probabilistic model of the solution space instead of an explicit population. Such a model is able to implicitly represent a solution set that is far larger than any realistic population size.

To support the study of how optimization algorithms handle large sets of optima, we propose the test function EqalBlocksOneMax (EBOM). It has an easy to optimize fitness landscape, however, with an exponential number of optima. We show that the bivariate EDA mutual-information-maximizing input clustering (MIMIC), without any problem-specific modification, quickly generates a model that behaves very similarly to a theoretically ideal model for that function, which samples each of the exponentially many optima with the same maximal probability.

References

[1]

Ignasi Belda, Sergio Madurga, Teresa Tarragó, Xavier Llorà, and Ernest Giralt. 2007. Evolutionary computation and multimodal search: A good combination to tackle molecular diversity in the field of peptide design. Molecular Diversity 11 (2007), 7--21.

[2]

Jeremy S. De Bonet, Charles Lee Isbell Jr., and Paul A. Viola. 1996. MIMIC: finding optima by estimating probability densities. In Proc. of NIPS '96. 424--430. http://papers.nips.cc/paper/1328-mimic-finding-optima-by-estimating-probability-densities

[3]

Chung-Yao Chuang and Wen-Lian Hsu. 2010. Multivariate multi-model approach for globally multimodal problems. In Proc. of GECCO '10. 311--318.

Digital Library

[4]

Kenneth Alan De Jong. 1975. An analysis of the behavior of a class of genetic adaptive systems. Ph.D. Dissertation. USA. University of Michigan.

[5]

Benjamin Doerr and Martin S. Krejca. 2020. The univariate marginal distribution algorithm copes well with deception and epistasis. In Proc. of EvoCOP '20. To appear.

[6]

Benjamin Doerr and Weijie Zheng. 2019. Sharp bounds for genetic drift in EDAs. CoRR abs/1910.14389 (2019). https://arxiv.org/abs/1910.14389

[7]

Tobias Friedrich, Timo Kötzing, and Martin S. Krejca. 2016. EDAs cannot be balanced and stable. In Proc. of GECCO '16. 1139--1146.

Digital Library

[8]

Georges R. Harik., Fernando G. Lobo, and David E. Goldberg. 1999. The compact genetic algorithm. IEEE Transactions on Evolutionary Computations 3, 4 (1999), 287--297.

Digital Library

[9]

Georges R. Harik, Fernando G. Lobo, and Kumara Sastry. 2006. Linkage learning via probabilistic modeling in the extended compact genetic algorithm (ECGA). In Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications. Springer, 39--61.

[10]

Mark Hauschild, Martin Pelikan, Claudio F. Lima, and Kumara Sastry. 2007. Analyzing probabilistic models in hierarchical BOA on traps and spin glasses. In Proc. of GECCO '07. 523--530.

Digital Library

[11]

Cem Hocaoǧlu and Arthur C. Sanderson. 1997. Multimodal function optimization using minimal representation size clustering and its application to planning multipaths. Evolutionary Computation 5, 1 (1997), 81--104.

Digital Library

[12]

Daphne Koller and Nir Friedman. 2009. Probabilistic graphical models: principles and techniques. The MIT Press.

Digital Library

[13]

Martin Krejca and Carsten Witt. 2020. Theory of estimation-of-distribution algorithms. In Theory of Evolutionary Computation: Recent Developments in Discrete Optimization. Springer, Chapter 9, 405--442. available online under https://arxiv.org/abs/1806.05392.

[14]

Per Kristian Lehre and Phan Trung Hai Nguyen. 2019. On the limitations of the univariate marginal distribution algorithm to deception and where bivariate EDAs might help. In Proc. of FOGA '19. 154--168.

Digital Library

[15]

Samir W. Mahfoud. 1996. Niching methods for genetic algorithms. Ph.D. Dissertation. USA. University of Illinois at Urbana-Champaign.

Digital Library

[16]

Heinz Mühlenbein and Gerhard Paaß. 1996. From recombination of genes to the estimation of distributions I. Binary parameters. In Proc. of PPSN IV. 178--187.

[17]

Brad L. Miller and Michael J. Shaw. 1996. Genetic algorithms with dynamic niche sharing for multimodal function optimization. In Proc. of CEC '96. 786--791.

[18]

José Peña, Jose Lozano, and Pedro Larranaga. 2005. Globally multimodal problem optimization via an estimation of distribution algorithm based on unsupervised learning of Bayesian networks. Evolutionary Computation 13 (2005), 43--66.

Digital Library

[19]

Martin Pelikan and David E. Goldberg. 2001. Escaping hierarchical traps with competent genetic algorithms. In Proc. of GECCO '01. 511--518. https://dl.acm.org/doi/10.5555/2955239.2955318

[20]

Martin Pelikan and David E. Goldberg. 2003. Hierarchical BOA solves Ising spin glasses and MAXSAT. In Proc. of GECCO '03. 1271--1282.

[21]

Martin Pelikan, Mark Hauschild, and Fernando G. Lobo. 2015. Estimation of distribution algorithms. In Springer Handbook of Computational Intelligence. Springer, 899--928.

[22]

Martin Pelikan and Heinz Mühlenbein. 1999. The bivariate marginal distribution algorithm. In Advances in Soft Computing. Springer, 521--535.

[23]

Gulshan Singh and Kalyanmoy Deb. 2006. Comparison of multi-modal optimization algorithms based on evolutionary algorithms. In Proc. of GECCO '06. 1305--1312.

Digital Library

Cited By

Doerr BLi XHandl J(2024)A Gentle Introduction to Theory (for Non-Theoreticians)Proceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3648402(800-829)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3648402
Doerr BSilva SPaquete L(2023)A Gentle Introduction to Theory (for Non-Theoreticians)Proceedings of the Companion Conference on Genetic and Evolutionary Computation10.1145/3583133.3595042(946-975)Online publication date: 15-Jul-2023
https://dl.acm.org/doi/10.1145/3583133.3595042
Doerr BKrejca M(2023)Bivariate estimation-of-distribution algorithms can find an exponential number of optimaTheoretical Computer Science10.1016/j.tcs.2023.114074971:COnline publication date: 6-Sep-2023
https://dl.acm.org/doi/10.1016/j.tcs.2023.114074
Show More Cited By

Index Terms

Bivariate estimation-of-distribution algorithms can find an exponential number of optima
1. General and reference
  1. Cross-computing tools and techniques
    1. Empirical studies
    2. Experimentation

Recommendations

On crossing fitness valleys with majority-vote crossover and estimation-of-distribution algorithms
FOGA '21: Proceedings of the 16th ACM/SIGEVO Conference on Foundations of Genetic Algorithms

The benefits of using crossover in crossing fitness gaps have been studied extensively in evolutionary computation. Recent runtime results show that majority-vote crossover is particularly efficient at optimizing the well-known Jump benchmark function ...
Bivariate estimation-of-distribution algorithms can find an exponential number of optima
Abstract
Finding a large set of optima in a multimodal optimization landscape is a challenging task. Classical population-based evolutionary algorithms typically converge only to a single solution. While this can be counteracted by applying ...
BBOB-benchmarking a simple estimation of distribution algorithm with cauchy distribution
GECCO '09: Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers

The restarted estimation of distribution algorithm (EDA) with Cauchy distribution as the probabilistic model is tested on the BBOB 2009 testbed. These tests prove that when using the Cauchy distribution and suitably chosen variance enlargment factor, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference

June 2020

1349 pages

ISBN:9781450371285

DOI:10.1145/3377930

General Chair:
Carlos Artemio Coello Coello
CINVESTAV-IPN

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Cooperation in Science and Technology
Investissement d'avenir project in a joint call with Gaspard Monge Program for optimization, operations research and their interactions with data sciences

Conference

GECCO '20

Sponsor:

SIGEVO

GECCO '20: Genetic and Evolutionary Computation Conference

July 8 - 12, 2020

Cancún, Mexico

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
85
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Doerr BLi XHandl J(2024)A Gentle Introduction to Theory (for Non-Theoreticians)Proceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3648402(800-829)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3648402
Doerr BSilva SPaquete L(2023)A Gentle Introduction to Theory (for Non-Theoreticians)Proceedings of the Companion Conference on Genetic and Evolutionary Computation10.1145/3583133.3595042(946-975)Online publication date: 15-Jul-2023
https://dl.acm.org/doi/10.1145/3583133.3595042
Doerr BKrejca M(2023)Bivariate estimation-of-distribution algorithms can find an exponential number of optimaTheoretical Computer Science10.1016/j.tcs.2023.114074971:COnline publication date: 6-Sep-2023
https://dl.acm.org/doi/10.1016/j.tcs.2023.114074
Doerr BWagner M(2022)A gentle introduction to theory (for non-theoreticians)Proceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3520304.3533628(890-921)Online publication date: 9-Jul-2022
https://dl.acm.org/doi/10.1145/3520304.3533628
Doerr BDufay M(2022)General Univariate Estimation-of-Distribution AlgorithmsParallel Problem Solving from Nature – PPSN XVII10.1007/978-3-031-14721-0_33(470-484)Online publication date: 10-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-14721-0_33
Doerr BKrawiec K(2021)A gentle introduction to theory (for non-theoreticians)Proceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3449726.3461406(369-398)Online publication date: 7-Jul-2021
https://dl.acm.org/doi/10.1145/3449726.3461406
Doerr BCoello Coello C(2020)A gentle introduction to theory (for non-theoreticians)Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion10.1145/3377929.3389889(373-403)Online publication date: 8-Jul-2020
https://dl.acm.org/doi/10.1145/3377929.3389889
Lin ZSu QXie G(2020)NMIEDA: Estimation of distribution algorithm based on normalized mutual informationConcurrency and Computation: Practice and Experience10.1002/cpe.607433:6Online publication date: 12-Nov-2020
https://doi.org/10.1002/cpe.6074

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten