Link to competitions – Benchmarks for Evaluation of Evolutionary Algorithms
P. N. Suganthan’s Homepage:
Research on single objective optimization algorithms often forms the foundation for more complex methods, such as niching algorithms and both multi-objective and constrained optimization algorithms. Traditionally, single objective benchmark problems are also the first test for new evolutionary and swarm algorithms. Additionally, single objective benchmark problems can be transformed into dynamic, niching composition, computationally expensive, and many other classes of problems. It is with the goal of better understanding the behavior of evolutionary algorithms as single objective optimizers that we are introducing the 100-Digit Challenge.
The original SIAM 100-Digit Challenge was developed in 2002 by Oxford’s Nick Trefethen in conjunction with the Society for Industrial and Applied Mathematics (SIAM) as a test for high-accuracy computing. Specifically, the challenge was to solve 10 hard problems to 10 digits of accuracy. One point was awarded for each correct digit, making the maximum score 100, hence the name. Contestants were allowed to apply any method to any problem and take as long as needed to solve it. Out of the 94 teams that entered, 20 scored 100 points and 5 others scored 99.
Like the SIAM version, our competition has 10 problems, which in our case are 10 functions to optimize, and the goal is to compute each function’s minimum value to 10 digits of accuracy without being limited by time. In contrast to the SIAM version, however, our 100-Digit Challenge asks contestants to solve all ten problems with one algorithm, although limited control parameter “tuning” for each function will be permitted to restore some of the original contest’s flexibility. Another difference is that the score for a given function is the average number of correct digits in the best 25 out of 50 trials (still a maximum of 10 points per function).
Five Other Numerical Optimization Competition Types
The details of other competitions on many-/multi-objective, multimodal, constrained single objective and single objective expensive problems can be found below.
If you face any difficulties or if you have suggestions to improve the technical report or if you find any potential bug in the codes, please inform Ponnuthurai Nagaratnam Suganthan (firstname.lastname@example.org).
- K. V. Price, N. H. Awad, M. Z. Ali, P. N. Suganthan, “Problem Definitions and Evaluation Criteria for the 100-Digit Challenge Special Session and Competition on Single Objective Numerical Optimization,” Technical Report, Nanyang Technological University, Singapore, November 2018. (Single objective bound constrained case)
- A bug fixed in problem F3 of 100-digit challenge on 17th Dec 2018.
- N. H. Awad, M. Z. Ali, J. J. Liang, B. Y. Qu and P. N. Suganthan, “Problem Definitions and Evaluation Criteria for the CEC 2017 Special Session and Competition on Single Objective Bound Constrained Real-Parameter Numerical Optimization,” Technical Report, Nanyang Technological University, Singapore, November 2016. (Single objective bound constrained case)
- Guohua Wu, R. Mallipeddi, P. N. Suganthan, “Problem Definitions and Evaluation Criteria for the CEC 2017 Competition and Special Session on Constrained Single Objective Real-Parameter Optimization“, Technical Report, Nanyang Technological University, Singapore, November 2016. (Single objective, constrained case)
- H Li, K Deb, Q Zhang, PN Suganthan, L Chen, “Comparison between MOEA/D and NSGA-III on a set of novel many and multi-objective benchmark problems with challenging difficulties,” Swarm and Evolutionary Computation, 2019. (Many and multi-objective case)
- Q. Chen, B. Liu, Q. Zhang, J. J. Liang, P. N. Suganthan, B. Y. Qu, “Problem Definition and Evaluation Criteria for CEC 2015 Special Session and Competition on Bound Constrained Single-Objective Computationally Expensive Numerical Optimization“, Technical Report, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou, China and Technical Report, Nanyang Technological University, Singapore, Nov 2014. (Computationally expensive case)
- B. Y. Qu, J. J. Liang, Z. Y. Wang, Q. Chen, P. N. Suganthan, “Novel Benchmark Functions for Continuous Multimodal Optimization with Comparative Results,” Swarm and Evolutionary Computation, doi:10.1016/j.swevo.2015.07.003, 2016.
Software and Data: Click on any of the titles of the technical reports in #1 to #6 to obtain the codes.