A COMPARISON OF OPTIMIZATION ALGORITHMS FOR STANDARD BENCHMARK FUNCTIONS

: Optimization algorithms are the search methods that are inspired from the natural biological evolution and social behavior of animals, insects, birds and humans, etc. The need of introducing the optimization algorithm is to achieve a near optimal solutions for complex and non-linear problems for which numerical methods may fail. This paper compares the performance of ten popular optimization algorithms: Grasshopper Optimization Algorithm, Adaptive Cuckoo Search Algorithm, Bird Swarm Algorithm, Chaotic Ant Lion Optimization Algorithm, Adaptive Wind Driven Algorithm, Real Coded Genetic Algorithm, Particle Swarm Optimization, Teaching Learning Based Optimization, Whale Optimization Algorithm and Ant Colony Optimization. A performance comparison of these optimization algorithms has been made on basis of accuracy by using five standard benchmark functions and it has been found that Whale Optimization Algorithm is performing better than other algorithms.


I INTRODUCTION
Optimization problems are solved by minimizing or maximizing the objective function using optimization algorithms. Optimization algorithms are divided into two categories: numerical methods and population based heuristic methods. Numerical methods solve the problem in finite time for a small search space [2]. Difficulties come in solving complex and large scale optimization problems using numerical methods gives chance to introduce the population based heuristic methods with high potential that can solve hard and complex engineering problems [1]. The population based optimization algorithms obtain the near optimum solution and provide better results than numerical methods in complex or hard engineering problems [3]. Optimization algorithms are based on the social behavior and biological evolution of animals, birds, insects, etc. Example includes the Grasshopper Optimization Algorithm (GOA) inspired from the attacking behavior of grasshoppers on crops [4], Whale Optimization Algorithm (WOA) influenced from hunting behavior of humpback whales to attack on a small herd of fish [12], Particle Swarm Optimization (PSO) influenced from foraging behavior of birds [10] and Genetic Algorithm (GA) inspired from the evolution of generations of human beings [5], etc. The behavior of these algorithms is based on evolution, learning and adaption process [4]. These algorithms have great potential and seek robust and fast solutions for complex and hard engineering problem. Various modified and adaptive optimization algorithms have been introduced by the researchers in the recent years which are used for solving different hard and complex engineering problems due to their high potential [1]. A performance comparison of popular optimization algorithms has been made in this work by using five benchmark functions on the basis of accuracy. Accuracy of optimization algorithms has been calculated by analyzing the results provided by optimization algorithms that how much these are closer to the global solution.

II OPTIMIZATION ALGORITHMS
The optimization algorithms used in this work are population based heuristic algorithms [13]. These algorithms minimize or maximize the objective function by validating the extant constraints [2]. A brief introduction of popular optimization algorithms is given below:

A. Grasshopper Optimization Algorithm:
The inspiration of Grasshopper Optimization Algorithm (GOA) comes from the attacking strategy of grasshoppers on crops and proposed by Shahrzad Saremi, Seyedali Mirjalili and Andrew Lewis [4]. Grasshoppers make attacks on the crops in the form of a swarm. They manage themselves according to the gravitational and wind force so that these factors become useful for them to attack on the crops or their target. Their social interaction among the swarm also plays an important role to make a successful attack. In this algorithm, values of cmax and cmin have been taken as 1 and 0.00004 respectively.

B. Real Coded Genetic Algorithm:
Genetic Algorithm (GA) algorithm is proposed by John Holland and inspired from the natural evolution of human beings [5]. In Real coded Genetic algorithm individuals are generated in the real number form instead of binary numbers. It is evolution or population based algorithm used for solving the optimization problems. In this algorithm, first initial population is randomly generated and then different operations (i.e. selection, crossover and mutation) are used to optimize the population for getting the best results. In selection, individuals are selected on the basis of their fitness value by using various selection methods like random, tournament, rank based and roulette wheel, etc.
Selected individuals are used for the next generation or iteration. In crossover, two random individuals are selected and a new individual is generated from the combined features or properties of both individuals. In mutation, the individuals are mutated by changing their some properties or features and that mutated individual are taken ahead which gives best fitness value. In this algorithm, values of crossover percentage and mutation percentage have been taken as 0.7 and 0.3 respectively.

C. Chaotic Ant Lion Optimization:
Chaotic Ant Lion Optimization is proposed by Hossam M. Zawbaa. It is the improved version of Ant Lion Optimization (ALO) algorithm [6]. The ALO algorithm is introduced by Sayedali Mirjalili and Shahrzad Saremi and inspired from the hunting behavior of Ant lions (Doodlebugs). For hunting the insects, ant lions dig a conical hole in the sand by using their huge jaws and hide themselves at the bottom of that hole. When an insect trapped into that hole then ant lions move sand towards the edge of the hole so that their prey would slide towards the bottom of the hole and they can catch it easily. In CALO, chaos means a condition or place with great confusion that makes the algorithm more capable of making random searches and escaping from local optima.

D. Adaptive Wind Driven Algorithm:
AWDO is introduced by Zikri Bayraktar and inspired from the path followed by a parcel moving in the space or atmosphere under the influence of various forces like wind, pressure and etc. [7]. In Adaptive WDO, values of parameters are optimized by using the Covariance Matrix Adaptive Evolutionary Strategy (CMAES) and these values are further used for updating the velocity of air parcels. In this algorithm, value of maximum speed of air parcel has been taken as 0.3.

E. Bird Swarm Algorithm:
It is a swarm based optimization algorithm proposed by Xian-Bing Meng. This algorithm is inspired from the social behaviour and interaction of birds in swarm [8]. Mostly, the birds move from one place to another in the search of food and for protecting themselves from predators or enemies. Therefore, their behaviour is divided into three categories in which birds perform different tasks as per their need. In Foraging Behaviour, birds will make their search for food. In Vigilance Behaviour, birds will keep vigilance of their enemies or predator to protect their swarm from them. In Flight Behaviour, birds fly to another place in search of food or for protecting themselves from any predator. In this algorithm, values of FQ (Frequency for flight behavior of birds) =10, c1 (coginitive accelerated coefficient) =1.5, c2 (social accelerated coefficient) =1.5, a and a2 (constants to make effect on vigilance behavior) =1 have been taken.

F. Adaptive Cuckoo Search Algorithm:
Adaptive Cuckoo Search (ACS) algorithm is invented by Xin-she Yang and Suash Deb and inspired from the behaviour of cuckoo birds [9]. Cuckoo birds give their eggs in the nest of other birds. Therefore, they find the best nest for their eggs. In ACS, Cuckoo is called as the nest, for defining the position of cuckoos in the search space. In the simple Cuckoo Search algorithm, the step size is not adjustable which is improved in ACS by adjusting the step size according to the fitness of the nest over the iterations. This helps the algorithm to more explore and search for the global solution. In this algorithm, values of abandoned probability and levy coefficient have been taken as 0.25 and 1.5 respectively.

G. Particle Swarm Optimization:
The inspiration of Particle Swarm Optimization (PSO) algorithm comes from the flocking behaviour of birds for searching the food and it is invented by Russell Eberhart and James Kennedy [10]. In PSO, birds are called as particles. At the initial stage, all particles search randomly for the food. The flock achieves the best position by communicating with other birds of the flock that already have a better position. The bird which has a better solution or close to the solution will inform it to its flock and the other birds will move towards that place. This would happen repeatedly until the best solution is achieved. In PSO, velocity of particles towards the best solution is updated on the basis of two best solutions: The knowledge of the environment (its fitness value) and the swarm's previous history of states. In this algorithm, values of inertia weight and damping ratio of inertia weight have been taken as 1 and 0.99 respectively.

H. Ant Colony Optimization:
It is introduced by Marco Dorigo and inspired from the stigmergy communication that ants use to communicate between them so that all the ants can achieve an optimized route to the food source [11]. There are main two characteristics of stigmergy communication: First, it is an indirect and non-symbolic form of communication in which environment is mediated means for communicating with each other ants modify their environment. Second, it is local and accessed by only that ant which comes in the locus of that area in which information is released. Ants live in the form of groups or colonies. They follow different paths in the search of food and leaves pheromone on the path. This pheromone is received by other ants and when the concentration of pheromone on one path becomes higher. Then all the ants shift towards that path over some time and achieve an optimized path to their food source. In this algorithm, values of tau (intial pheromone) =0.008, alpha (Pheromone exponential weight) =1, beta (Heuristic exponential weight) =1 and rho (Evaporation rate) =0.005 have been taken.

I. Whale Optimization Algorithm:
The Whale Optimization Algorithm (WOA) is introduced by Seyedali Mirjalili and inspired from the Bubble-net feeding behaviour of humpback whales [12]. The bubble net feeding behaviour is basically a foraging or hunting method of humpback whales. Mainly whales live in groups and their favourite food is herd of small fishes. For hunting small fishes they follow the bubble net feeding behaviour. In this method whales dive 12m down from the surface of water and start moving up and swim towards the surface of the water in the shape of 9 or in the spiral movement around their food to get it.

J. Teaching Learning Based Optimization:
Teaching Learning Based Optimization (TLBO) is introduced by Ravipudi Venkata Rao and inspired from the learning process of classroom where students learn from a teacher and from themselves by discussing or sharing knowledge [13]. This algorithm works in two phases: Teacher Phase and Learner Phase. In Teacher Phase, the best solution of the current iteration is selected as a teacher and updates the knowledge of all students towards teacher by using their mean. In the learner phase, two random students of current iteration are taken from the population and they update the knowledge of each other by sharing the knowledge among themselves.

III OPTIMIZATION TESTING FUNCTIONS
Optimization testing functions are used for testing the performance of optimization techniques that how they perform on different type of problems. In this work, the performance of ten popular optimization algorithms has been compared by using the five benchmark optimization functions. The applied functions have been widely used in testing the various optimization techniques. The Ackley function is a multimodal function with a large number of local optima [14]. The Rastrigin function is a non-linear, multimodal and NP hard problem [18]. The Griewank is a widespread local optima function [19]. The Sphere is a continuous, convex, unimodal and NP hard problem [20]. The Rosenbrock is a multimodal function with a narrow valley and it is very difficult to find its global optimum [17]. The applied optimization testing functions are described below:

A. Ackley Function:
The Ackley function is widely used for testing optimization algorithms. It is a multimodal test function. In its twodimensional form, it is described with nearly flat outer region and a large hole at the center as shown in Figure 1. The function creates a trouble for optimization algorithms to be trapped in one of its many local minima. The Ackley function is defined as in (1):

B. Rastrigin Function:
The Rastrigin function is widely used as optimization testing function. It is a multimodal function and has many local minima, but the locations of minima are distributed regularly. Figure 2 shows a plot of Rastrigin function in two-dimensional form. The Rastrigin Function is defined as in (2): The function is usually evaluated in the range of x i ∈ [-5.12, 5.12], for all i = 1, …, d and its global minima is f(x) = 0 at x = 0 [14].

C. Griewank Function:
The Griewank function is a widely used optimizing testing function. It has many widespread local minima, which are regularly distributed. The complexity of the Griewank function is shown in the zoomed-in plot. Figure 3 The Griewank function is defined as in (3): The function is usually evaluated in the range of x i ∈ [-600, 600], for all i = 1, …, d and its global minimum is f(x) = 0 at x = 0 [15].

D. Sphere Function:
The Sphere function is a popular optimization testing function. It has many local minima except for the global one. It is continuous, unimodal and convex function. Figure  4 shows a plot of the two-dimensional Sphere function. The Sphere function is defined as in (4): The Sphere function is usually evaluated in the range of x i ∈ [-5.12, 5.12], for all i = 1,…, d and its global minimum is at f(x) = 0 at x = 0 [16].

E. Rosenbrock Function:
The Rosenbrock function is a popular optimization testing function. It is also called as Banana or Valley function. It is a unimodal function and its global minimum lies in a narrow and parabolic valley. However, it is easy to find the valley as the convergence to the minimum is very difficult. Figure  5 shows a plot of Rosenbrock function in two dimensional form. The Rosenbrock function is defined as in (5): The function is usually evaluated in the range of x i ∈ [-2.048, 2.048], for all i = 1, …, d and its global minimum is f(x) = 0 at x = 0 [17]. Table I

V CONCLUSION
In this paper, ten popular optimization algorithms have been chosen based on the popularity and five benchmark functions have been used for testing. A brief introduction to each of the algorithms and the benchmark functions is presented. Then a comparison of these algorithms has been made by using these five benchmark functions. From performance comparison of the applied optimization algorithms, it has been concluded that Whale Optimization Algorithm is performing better than the others as it is providing global optimal solutions on Rastrigin and Griewank functions and good results on other three functions. In future, these techniques will be tested on the other benchmark function.