research-article

On crossing fitness valleys with majority-vote crossover and estimation-of-distribution algorithms

Author:

Carsten WittAuthors Info & Claims

FOGA '21: Proceedings of the 16th ACM/SIGEVO Conference on Foundations of Genetic Algorithms

Article No.: 2, Pages 1 - 15

https://doi.org/10.1145/3450218.3477303

Published: 06 September 2021 Publication History

Abstract

The benefits of using crossover in crossing fitness gaps have been studied extensively in evolutionary computation. Recent runtime results show that majority-vote crossover is particularly efficient at optimizing the well-known Jump benchmark function that includes a fitness gap next to the global optimum. Also estimation-of-distribution algorithms (EDAs), which use an implicit crossover, are much more efficient on Jump than typical mutation-based algorithms. However, the allowed gap size for polynomial runtimes with EDAs is at most logarithmic in the problem dimension n.

In this paper, we investigate variants of the Jump function where the gap is shifted and appears in the middle of the typical search trajectory. Such gaps can still be overcome efficiently in time O (n log n) by majority-vote crossover and an estimation-of-distribution algorithm, even for gap sizes almost [EQUATION]. However, if the global optimum is located in the gap instead of the usual all-ones string, majority-vote crossover would nevertheless approach the all-ones string and be highly inefficient. In sharp contrast, an EDA can still find such a shifted optimum efficiently. Thanks to a general property called fair sampling, the EDA will with high probability sample from almost every fitness level of the function, including levels in the gap, and sample the global optimum even though the overall search trajectory points towards the all-ones string. Finally, we derive limits on the gap size allowing efficient runtimes for the EDA.

Supplementary Material

MP4 File (foga21-video4.mp4)

presentation at FOGA'21 by C. Witt on the paper "On crossing fitness valleys with majority-vote crossover and estimation-of-distribution algorithms"

Download
617.47 MB

References

[1]

David Ackley. A Connectionist Machine for Genetic Hillclimbing. Springer, 1987.

[2]

Jean-Bernard Baillon, Roberto Cominetti, and José Vaisman. A sharp uniform bound for the distribution of sums of bernoulli trials. Combinatorics, Probability and Computing, 25(3):352--361, 2016.

[3]

Henry Bambury, Antoine Bultel, and Benjamin Doerr. An extended jump function benchmark for the analysis of randomized search heuristics. In Proc. of GECCO '21, page 1124--1132. ACM Press, 2021.

Digital Library

[4]

Riade Benbaki, Ziyad Benmar, and Benjamin Doerr. A rigorous runtime analysis of the 2-MMAS_ib on jump functions: Ant colony optimizers can cope well with local optima. In Proc. of GECCO '21, pages 4--13. ACM Press, 2021.

Digital Library

[5]

Dogan Corus, Pietro S. Oliveto, and Donya Yazdani. On the runtime analysis of the Opt-IA artificial immune system. In Proc. of GECCO '17, pages 83--90. ACM Press, 2017.

Digital Library

[6]

Dogan Corus, Pietro S. Oliveto, and Donya Yazdani. Fast artificial immune systems. In Proc. of PPSN '18, page 67--78. Springer, 2018.

[7]

Duc-Cuong Dang, Tobias Friedrich, Timo Kötzing, Martin S. Krejca, Per Kristian Lehre, Pietro S. Oliveto, Dirk Sudholt, and Andrew M. Sutton. Escaping local optima using crossover with emergent diversity. IEEE Transactions on Evolutionary Computation, 22(3):484--497, 2018.

[8]

Benjamin Doerr. The runtime of the compact genetic algorithm on jump functions. arXiv, abs/1908.06527, 2019. URL https://arxiv.org/abs/1908.06527; revised preprint combining the results of the following two entries.

[9]

Benjamin Doerr. An exponential lower bound for the runtime of the compact genetic algorithm on jump functions. In Proc. of FOGA '19, pages 25--33. ACM Press, 2019.

Digital Library

[10]

Benjamin Doerr. A tight runtime analysis for the cGA on jump functions: EDAs can cross fitness valleys at no extra cost. In Proc. of GECCO '19, pages 1488--1496. ACM Press, 2019.

Digital Library

[11]

Benjamin Doerr. Probabilistic tools for the analysis of randomized optimization heuristics. In Benjamin Doerr and Frank Neumann, editors, Theory of Evolutionary Computation - Recent Developments in Discrete Optimization, pages 1--87. Springer, 2020.

[12]

Benjamin Doerr and Weijie Zheng. Sharp bounds for genetic drift in estimation of distribution algorithms. IEEE Transactions on Evolutionary Computation, 24(6): 1140--1149, 2020.

Digital Library

[13]

Benjamin Doerr, Huu Phuoc Le, Régis Makhmara, and Ta Duy Nguyen. Fast genetic algorithms. In Proc. of GECCO '17, pages 777--784. ACM Press, 2017.

Digital Library

[14]

Stefan Droste. A rigorous analysis of the compact genetic algorithm for linear functions. Natural Computing, 5(3):257--283, 2006.

Digital Library

[15]

Stefan Droste, Thomas Jansen, and Ingo Wegener. On the analysis of the (1+1) evolutionary algorithm. Theoretical Computer Science, 276:51--81, 2002.

Digital Library

[16]

Stefan Droste, Thomas Jansen, and Ingo Wegener. Upper and lower bounds for randomized search heuristics in black-box optimization. Theory of Computing Systems, 39:525--544, 2006.

Digital Library

[17]

Tobias Friedrich, Timo Kötzing, Martin S. Krejca, Samadhi Nallaperuma, Frank Neumann, and Martin Schirneck. Fast building block assembly by majority vote crossover. In Proc. of GECCO '16, pages 661--668. ACM Press, 2016.

Digital Library

[18]

Tobias Friedrich, Timo Kötzing, Martin S. Krejca, and Andrew M. Sutton. The compact genetic algorithm is efficient under extreme gaussian noise. IEEE Transactions on Evolutionary Computation, 21(3):477--490, 2017.

Digital Library

[19]

Tobias Friedrich, Francesco Quinzan, and Markus Wagner. Escaping large deceptive basins of attraction with heavy-tailed mutation operators. In Proc. of GECCO '18, pages 293--300. ACM Press, 2018.

Digital Library

[20]

Leon Jay Gleser. On the distribution of the number of successes in independent trials. The Annals of Probability, 3(1):182 -- 188, 1975.

[21]

Georges R. Harik, Fernando G. Lobo, and David E. Goldberg. The compact genetic algorithm. IEEE Transactions on Evolutionary Computation, 3(4):287--297, 1999.

Digital Library

[22]

Wassily Hoeffding. On the distribution of the number of successes in independent trials. The Annals of Mathematical Statistics, 27(3):713 -- 721, 1956.

[23]

Hsien-Kuei Hwang and Carsten Witt. Sharp bounds on the runtime of the (1+1) EA via drift analysis and analytic combinatorial tools. In Proc. of FOGA '19, pages 1--12. ACM Press, 2019.

Digital Library

[24]

Thomas Jansen. On the black-box complexity of example functions: The real jump function. In Proc. of FOGA '15, pages 16--24. ACM Press, 2015.

Digital Library

[25]

Thomas Jansen and Ingo Wegener. The analysis of evolutionary algorithms - A proof that crossover really can help. Algorithmica, 34(1):47--66, 2002.

Digital Library

[26]

Timo Kötzing, Dirk Sudholt, and Madeleine Theile. How crossover helps in pseudo-boolean optimization. In Proc. of GECCO 2011, pages 989--996. ACM Press, 2011.

Digital Library

[27]

Per Kristian Lehre and Carsten Witt. Tail bounds on hitting times of randomized search heuristics using variable drift analysis. Combinatorics, Probability & Computing, 30(4):550--569, 2021.

[28]

Johannes Lengler. Drift analysis. In Benjamin Doerr and Frank Neumann, editors, Theory of Evolutionary Computation - Recent Developments in Discrete Optimization, pages 89--131. Springer, 2020.

[29]

Johannes Lengler, Dirk Sudholt, and Carsten Witt. The complex parameter landscape of the compact genetic algorithm. Algorithmica, 83(4):1096--1137, 2021.

Digital Library

[30]

Andrei Lissovoi, Pietro S. Oliveto, and John Alasdair Warwicker. On the time complexity of algorithm selection hyper-heuristics for multimodal optimisation. In Proc. of AAAI '19, pages 2322--2329. AAAI Press, 2019.

[31]

Albert W. Marshall, Ingram Olkin, and Barry C. Arnold. Inequalities: Theory of Majorization and Its Applications. Springer, 2nd edition, 2011.

[32]

Pietro S. Oliveto and Carsten Witt. Improved time complexity analysis of the simple genetic algorithm. Theoretical Computer Science, 605:21--41, 2015.

Digital Library

[33]

Adam Prügel-Bennett. Benefits of a population: Five mechanisms that advantage population-based algorithms. IEEE Transaction on Evolutionary Computation, 14(4):500--517, 2010.

Digital Library

[34]

Amirhossein Rajabi and Carsten Witt. Self-adjusting evolutionary algorithms for multimodal optimization. In Proc. of GECCO '20, pages 1314--1322. ACM Press, 2020.

Digital Library

[35]

Amirhossein Rajabi and Carsten Witt. Stagnation detection with randomized local search. In Proc. of EvoCOP '21, pages 152--168. Springer, 2021.

[36]

Amirhossein Rajabi and Carsten Witt. Stagnation detection in highly multimodal fitness landscapes. In Proc. of GECCO '21, page 1178--1186. ACM Press, 2021.

Digital Library

[37]

Jonathan E. Rowe and Aishwaryaprajna. The benefits and limitations of voting mechanisms in evolutionary optimisation. In Proc. of FOGA '19, pages 34--42. ACM Press, 2019.

[38]

Dirk Sudholt. How crossover speeds up building block assembly in genetic algorithms. Evolutionary Computation, 25(2):237--274, 2017.

Digital Library

[39]

Dirk Sudholt and Carsten Witt. On the choice of the update strength in estimation-of-distribution algorithms and ant colony optimization. Algorithmica, 81(4): 1450--1489, 2019.

Digital Library

[40]

Darrell Whitley, Swetha Varadarajan, Rachel Hirsch, and Anirban Mukhopadhyay. Exploration and exploitation without mutation: Solving the jump function in θ(n) time. In Proc. of PPSN '18, pages 55--66. Springer, 2018.

[41]

Carsten Witt. Domino convergence: why one should hill-climb on linear functions. In Proc. of GECCO '18, pages 1539--1546. ACM Press, 2018.

Digital Library

[42]

Carsten Witt. Upper bounds on the running time of the univariate marginal distribution algorithm on OneMax. Algorithmica, 81(2):632--667, 2019.

Digital Library

Cited By

Opris ALengler JSudholt DLi XHandl J(2024)A Tight O(4/p) Runtime Bound for a (μ+1)GA on Jump for Realistic Crossover ProbabilitiesProceedings of the Genetic and Evolutionary Computation Conference10.1145/3638529.3654120(1605-1613)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638529.3654120
Rajabi AWitt C(2024)Stagnation Detection in Highly Multimodal Fitness LandscapesAlgorithmica10.1007/s00453-024-01249-w86:9(2929-2958)Online publication date: 2-Jul-2024
https://doi.org/10.1007/s00453-024-01249-w
Doerr CKrejca M(2023)Run Time Analysis for Random Local Search on Generalized Majority FunctionsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.321634927:5(1385-1397)Online publication date: 3-Oct-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3216349
Show More Cited By

Index Terms

On crossing fitness valleys with majority-vote crossover and estimation-of-distribution algorithms
1. Theory of computation
  1. Theory and algorithms for application domains
    1. Theory of randomized search heuristics

Recommendations

How majority-vote crossover and estimation-of-distribution algorithms cope with fitness valleys
Abstract
The benefits of using crossover in crossing fitness gaps have been studied extensively in evolutionary computation. Recent runtime results show that majority-vote crossover is particularly efficient at optimizing the well-known Jump ...
Highlights
- We study 2 algorithms using majority-vote crossover and an estimation-of-distribution algorithm on modified jump functions.
When Non-Elitism Outperforms Elitism for Crossing Fitness Valleys
GECCO '16: Proceedings of the Genetic and Evolutionary Computation Conference 2016

Crossing fitness valleys is one of the major obstacles to function optimization. In this paper we investigate how the structure of the fitness valley, namely its depth d and length l, influence the runtime of different strategies for crossing these ...
Fast mutation in crossover-based algorithms
GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference

The heavy-tailed mutation operator proposed in Doerr et al. (GECCO 2017), called fast mutation to agree with the previously used language, so far was successfully used only in purely mutation-based algorithms. There, it can relieve the algorithm ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

FOGA '21: Proceedings of the 16th ACM/SIGEVO Conference on Foundations of Genetic Algorithms

September 2021

130 pages

ISBN:9781450383523

DOI:10.1145/3450218

General Chair:
Steffen Finck
Vorarlberg University of Applied Sciences
,
Program Chairs:
Michael Hellwig
Vorarlberg University of Applied Sciences
,
Pietro S. Oliveto
University of Sheffield

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Danish Council for Independent Research

Conference

FOGA '21

Sponsor:

SIGEVO

FOGA '21: Foundations of Genetic Algorithms XVI

September 6 - 8, 2021

Virtual Event, Austria

Acceptance Rates

FOGA '21 Paper Acceptance Rate 10 of 21 submissions, 48%;

Overall Acceptance Rate 72 of 131 submissions, 55%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
116
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Opris ALengler JSudholt DLi XHandl J(2024)A Tight O(4/p) Runtime Bound for a (μ+1)GA on Jump for Realistic Crossover ProbabilitiesProceedings of the Genetic and Evolutionary Computation Conference10.1145/3638529.3654120(1605-1613)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638529.3654120
Rajabi AWitt C(2024)Stagnation Detection in Highly Multimodal Fitness LandscapesAlgorithmica10.1007/s00453-024-01249-w86:9(2929-2958)Online publication date: 2-Jul-2024
https://doi.org/10.1007/s00453-024-01249-w
Doerr CKrejca M(2023)Run Time Analysis for Random Local Search on Generalized Majority FunctionsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.321634927:5(1385-1397)Online publication date: 3-Oct-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3216349
Antipov DBuzdalov MDoerr B(2023)Lazy Parameter Tuning and Control: Choosing All Parameters Randomly from a Power-Law DistributionAlgorithmica10.1007/s00453-023-01098-z86:2(442-484)Online publication date: 22-Feb-2023
https://dl.acm.org/doi/10.1007/s00453-023-01098-z
Ajimakin ADevi V(2022)The Competing Genes Evolutionary Algorithm: Avoiding Genetic Drift Through Competition, Local Search, and Majority VotingIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.322903827:6(1678-1689)Online publication date: 14-Dec-2022
https://dl.acm.org/doi/10.1109/TEVC.2022.3229038
Unanue IMerino MLozano J(2022)A mathematical analysis of EDAs with distance-based exponential modelsMemetic Computing10.1007/s12293-022-00371-y14:3(305-334)Online publication date: 16-Jul-2022
https://doi.org/10.1007/s12293-022-00371-y
Antipov DDoerr BKaravaev V(2022)A Rigorous Runtime Analysis of the GA on Jump FunctionsAlgorithmica10.1007/s00453-021-00907-784:6(1573-1602)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s00453-021-00907-7

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten