1 Introduction

Particle swarm optimization (PSO) is a metaheuristic optimization approach that imitates the dynamics of biological systems such as the swarm of birds. Kennedy and Eberhart [1] published the first seminal paper on PSO. As a result, the variety of research articles ascribed to the development of particle swarm optimization and swarm intelligence grown massively. The PSO algorithm is simple to implement and has a low memory requirements. Consequently, it has been used in a multitude of complex applications that involve optimization, control, machine learning, and so on.

PSO is widely used in the design, sizing, control and maximum power point tracking (MPPT) of renewable energy systems; such as photovoltaic (PV) systems [2,3,4], wind turbines [5,6,7] and Hybrid PV-wind systems [8,9,10]. The authors in [11,12,13] implemented PSO in the control of robotics for various applications. PSO is also popular in unmanned vehicles in performing path planning [14, 15] and path tracking [16,17,18].

PSO has been implemented widley in the sizing, design, modeling, and control of various typres of refrigeration system, such as reducing power consumption of multiple chiller systems [19], optimizing cascade refrigeration cycles [20], standing wave thermoacoustic refrigerators [21] and vapor compression refrigeration cycles [22].

Nevertheless, while PSO is efficient in optimizing multidimensional complexities [23], its drawbacks involve premature convergence and stagnation to local optima [24,25,26,27,28,29], as well as a slow convergence rate [24, 25, 28,29,30]. According to Jiao et al. and Chen [24, 25], the drawbacks of PSO originates from its own structure. The social model readily falls into local optima, practically, each particle in PSO follows rather dynamically the path of the global best position. This global best could be influenced by a local minimum, which leads all particles to a position with poor fitness. Whereas, the cognitive model has the drawback of slow convergence especially in the final phases of the evolving iterations when tackling various complexities. This motivates the particles’ classifications in order to treat each category with different updating strategy to achieve better convergence rates and to enhance global explorations capabilities [25, 31,32,33,34].

The authors in [24, 31, 35,36,37] implemented the particle elitism approach to speed up the convergence of PSO. Elitism is a process in which particles with poor fitness values are replaced with particles with the best fitness value (elite particles) after a certain number of iterations, resulting in the production of a new swarm with a better particle’s average fitness. The elitism process in nature requires a preliminary step which is the particle classification, in order to distinguish poor and elite particles. This process will indeed speed up the convergence but on the other hand, it will lessen the diversity of the particles and may increase the probability of falling into a local optimum.

In order to boost particle swarm diversity and mitigate the risk of falling into local optimum, references [24, 38,39,40,41] proposed the concept of mutation in PSO algorithm to improve the convergence accuracy and speed. In the literature there are mainly two mutation’s types implemented in the PSO algorithm. The first one is the global best mutation as proposed by [24, 41], in which the particles’ position are mutated indirectly in response to the change of global best. The second type is the direct position mutation as reported by [40], which is better emulating the mutation process inspired by genetic algorithms (GA). The mutation process will enhance the diversity of the swarm which results in better exploration capabilities but it may slow down the convergence as reported by [40].

Janson and Middendorf [42] introduced the concept of hierarchical particle swarm optimization (HPSO). Several authors [43,44,45] utilized and improved this method to improve the exploration and exploitation performance of the PSO. This approach arranges the particles into a dynamic hierarchy to construct a neighborhood structure. The particles move up or down the hierarchy based on the quality of their fitness. As a result, the higher fitness particles that advance up the hierarchy have a greater effect on the particle’s movements.

To address the above PSO’s disadvantages; slow convergence, loss of particles diversity, premature convergence, and stagnation to local optima. This work will propose a fast-converging targeted position-mutated elitism (TPME) operator that requires a fitness-based classification technique to categorize the particles. The introduced classification is motivated by its simplicity, low memory requirements, and automated termination criteria based on convergence. Initially, the basic PSO will be highlighted, along with several PSO variants that involve the three key elements affecting the convergence speed and global exploration capabilities: particle classification, elitism, and mutation. The proposed PSO method, particle swarm optimization with targeted position-mutated elitism (PSO-TPME), will benefit from these key features in order to enhance both convergence speed and global exploration capabilities by introducing an alternative classification technique, elitism, and targeted position mutation in the PSO algorithm. Following the No Free Lunch (NFL) theorem [46, 47], which establishes the risks of comparing algorithms based on their performance on a limited sample of problems, we will test the proposed PSO on thirteen multidimensional benchmark functions with various complexities in this study.

2 Related Works

Particle swarm optimization is efficient at optimizing multidimensional, complex applications. However, its drawbacks involve premature convergence, stagnation to local optima, [24,25,26,27,28,29], and a slow convergence rate [24, 25, 28,29,30]. Many PSO variants addressed the original PSO’s shortcomings. This work discusses the PSO variants incorporating bioinspired operators (mutation and elitism) that necessitate particle classification. The advantages and disadvantages of these operators were discussed in the introduction. It is worth noting that some PSO variants, including PSO-based feature selection [48, 49], effectively addressed the computational complexity of the PSO for extremely high-dimensional problems. This section illustrates basic PSO and several PSO variants incorporating mutation, elitism, and particle classification.

2.1 Particle Swarm Optimization (PSO)

PSO’s method generates a swarm of particles at random positions to represent possible solutions to an optimization task throughout the parameter space. The particle locations are hence iterated with the target of achieving a global optimum of a fitness value. The algorithm then examines each particle’s position and stores the best solution for each particle, which is called personal best (\(P_b\)). Moreover, during every iteration (It), PSO records the best solution throughout the swarm, which is called global best (\(G_b\)). For every subsequent iteration, the particles’ position (x) and velocity (v) are determined as a function of the swarm’s best position (social component), the particle’s best personal position (the cognitive component), and its prior velocity (memory component). In general, PSO has several versions, linearly decreasing weight particle swarm optimization is one of the basic versions of PSO, known as LDW-PSO, that reads [1, 50];

$$\begin{aligned} v_{ij}^{k+1}= & {} wv_{ij}^{k}+c_{1}r_{1}({P_b}_{ij}-x_{ij}^{k})+c_{2}r_{2}({G_b}_{j}-x_{ij}^{k}) \end{aligned}$$
(1)
$$x_{{ij}}^{{k + 1}} = x_{{ij}}^{k} + v_{{ij}}^{{k + 1}}$$
(2)
$$\begin{aligned} w= & {} w_{max}-It\times \frac{(w_{max}-w_{min})}{It_{max}} \end{aligned}$$
(3)

The subscript \(_i\) is ranging from 1 to N number of particles and the subscript \(_j\) is ranging from 1 to n dimensions. The factor w is known as the inertia weight, \(w_{max}\) and \(w_{min}\) are the maximum and the minimum inertia weight, respectively, and the product wv represents the particle’s momentum. The acceleration factors are \(c_1\) and \(c_2\), while \(r_1\) and \(r_2\) are the output of random generators ranging from 0 to 1.

2.2 PSO With Classification

Chen [25] proposed PSO-M, a study on particle classification that improves the convergence rate and convergence accuracy of PSO. The classification is based on observing the particles’ fitness value at each iteration, identifying the particles’ fitness mean (aver) and also the best and worst fitness values, (\(f_{max}\)) and (\(f_{min}\)), respectively. Then, two averages are calculated, aver1 and aver2, between \(f_{max}\) and aver and between \(f_{min}\) and aver, respectively. Hence, the fitness space (\(f_{min}\) – \(f_{max}\)) is divided into three categories. Considering maximization problem, the classification will be as follows; particles with fitness values in the range (aver1 – \(f_{max}\)) are updated using the cognitive PSO component, particles with fitness values in the range (\(f_{min}\) – aver2) are updated using the social PSO component, and the remaining particles are updated using the basic PSO model. One drawback of this classification is that all particles are classified starting from the initialization and can even last after the convergence, that is, no classification termination criteria, as seen in Fig. 1.

Fig. 1
figure 1

Particles’ evolution and classification with evolving iterations using PSO-M

Full size image

Rezaei and Safavi [34] introduced a guided adaptive search particle swarm optimization (GuASPSO). With this method, the personal best particles are all classified into clusters using a self-organizing map (SOM). With the evolving iterations, the number of these clusters is decreasing linearly. The weighted average calculated across the best particles of other clusters is used to get the unique global best guide of a specific particle. Due to the well-dispersed particles across the search space, the reasonable distance between each particle and its unique global best guide prevents the particles from becoming trapped in local optima, drifting, or having loss of diversity.

2.3 PSO with mutation

Influenced by genetic algorithms (GA), Jiao et al. [24] proposed the concept of elite PSO with mutation (EPSOM) to improve the convergence accuracy and speed. The global best was mutated to boost particle swarm diversity and mitigate the risk of falling into local optimum. The mutation is performed as follows;

$$\begin{aligned} {G_b}^{\prime }=G_b(1+0.5\eta ) \end{aligned}$$
(4)

where \(\eta\) is a randomly generated number ranging from 0 to 1.

Another recent study by Lü et al. [41] that relies on the global best mutation in a similar manner as in (4), proposed an adaptive weighted and mutation particle swarm optimization (AWMPSO) to enhance global search capabilities of the PSO. In which the mutation probability of the global best is now adaptive, where the mutation probability of the global best depend on the population fitness’s variance. Apart from the significant improvement of this approach on the enhanced exploration ability and the convergence speed and accuracy, this approach still has limitations due to the fact that the mutation is implemented on the global best as seen in (4). Hence, the particles’ positions are mutated indirectly through the global best mutation. According to (4), the change in the global best is limited and can vary gradually with the evolving iterations. Furthermore, the mutation is not targeted, since the mutation of the global best affect all particles. In this case, the mutation will indeed be useful to poor or moderate particles but it may also deteriorate the fitness of good particle.

Wang et al. [40] proposed a new method that relies on the position mutation (M-PSO). The particles’ position mutation process is activated conditionally as an intermediate step just before the evaluation of the personal best and the global of the PSO algorithm. The condition requires that a randomly generated number to be less than an adaptively generated threshold value (TH). The threshold value was defined as:

$$\begin{aligned} TH=\left( 1-\frac{i-1}{It_{max}-1}\right) ^{\frac{1}{mu}} \end{aligned}$$
(5)

where mu is the mutation factor. Wang et al. [40] reported that increasing mutation factor can enhance the convergence accuracy but it reduces the convergence speed. As seen in (5), the threshold value is updated by the evolving iterations, not the particles’ fitness, which can be mainly considered as a termination criterion. The threshold value is crucial parameter in M-PSO approach, not only because it is dictating the activation of the mutation process, but because it also decides the upper and the lower bounds of the mutated position. In M-PSO approach, there is no particle classifications, hence, the mutation is performed on all the particles. That is, M-PSO does not use targeted mutation which can contribute negatively on the movement of the good particles and consequently affect the convergence speed as reported by Wang et al. [40].

3 Proposed PSO (PSO-TPME)

The initial concept of the proposed PSO is automated-termination, adaptive particles’ classification with elitism based on particles’ fitness values (f). The classification is performed as follows; the mean (m) of the fitness values of all particles is calculated at each iteration. Then, a fixed percentage (p) around the mean is calculated to generate lower and upper bounds. Now the particles are divided into three categories: “good”, “fair”, and “bad”. Hence, considering a maximization problem, particles with fitness higher than the upper bound (“good” particles) can decrease their velocity to improve exploitation in the local domain via relying on their personal best (PSO cognitive component) instead of the global best (PSO social component). The particles with fitness located within the lower and upper bounds (“fair” particles) and having relatively average fitness can continue both exploration and exploitation while using the basic PSO algorithm. Particles with fitness less than the lower bound (“bad” particles) can initially increase their velocities to enhance exploration in the global search domain via relaying on the global best instead of their personal best. If the “bad” particles after several iterations \(N_{e}\) are still classified as “bad” particles, then the elitism process is activated to speed up the convergence rate, where \(N_{e}\) is the iteration number that initiates the targeted position-mutated elitism. The elitism process is intended to cope with the so-called “hopeless particles,” those given the chance to explore the search space and fail to level up to a better category. Now the elitism simply locates the hopeless particles in the position of the particle with the maximum fitness (\(f_{max}\)). This process speeds up the convergence rate significantly with proper choice of \(N_{e}\) as shown in the Fig. 2.

Fig. 2
figure 2

Particles’ evolution with evolving iterations using the proposed classification with elitism

Full size image

Conducting PSO using the earlier approach will drastically reduce the diversity of the particle swarm during the preliminary stages of evolving iterations. As a result, the probability of entrapment in a region of the local optimum is high. Adding a mutation to the particle’s position of the maximum fitness \((x_{j}(f_{max}))\) will boost the diversification of the elite particle and reduce the possibility of falling into a local optimum. Hence, the mutation is now targeting bad particles only, and the mutation is performed on the particle’s position directly instead of indirect mutation of the position using the global best as proposed by Jiao et al. [24]. The details of the proposed PSO algorithm considering a maximization problem are in (6), in which the elite particle’s positions \((x_{j}(f_{max}))\) are mutated by \((2a\eta +(1-a))\), where \(\eta\) is a randomly generated number ranging from 0 to 1, a and mp are presetting parameters that define the mutation range and mutation probability, respectively. The basic idea is that the elite position \((x_{j}(f_{max}))\) is mutated in the range \((x_{j}(f_{max})(1\pm a))\), in which the mean is unity multiplied by \(x_{j}(f_{max})\) which gives higher probability to \(x_{j}(f_{max})\), that is, exploitation of the elite particle’s position without neglecting the chances of exploring new particles’ positions. The aforementioned range can be increased or decreased via varying a for higher or lower dimension functions, respectively.

The proposed PSO is further explained in the pseudo code in algorithm 1 and the flowchart in Fig. 3. The algorithm’s inputs are an objective function with predetermined dimensions (n) and the total number of particles (N). The algorithm produces the global best (\(G_b\)) at the end of the iterations. Two “for loops” are used in the code; the first one iterates till a specific maximum number of iterations, whereas the second loop iterates over the number of particles. The results section will analyze the algorithm’s key parameters (a, mp, and N).

This classification is called automated-termination classification because, after some iterations, all the particles will fall into the middle category since, on one hand, the particles’ fitness values will eventually be close and, on the other hand, the particles’ mean fitness is gradually increasing which will expand the bounds of the middle category. Hence, the classification will stop automatically as depicted in Fig. 2. This classification is also considered adaptive classification because of particles’ mean variation at each iteration. The particles’ mean will change, therefore the upper and lower bound of the middle category will dynamically vary, as the calculation of these bounds depends only on a fixed percentage around the particles’ mean. This type of particles’ classification is motivated by its implantation simplicity, low memory requirements, and automated-termination criteria based on the particles convergence. The latter is very crucial since elitism with mutation takes part in the proposed PSO. Now, the automated termination criteria will stop the classification and the elitism with mutation as well, without manual or other termination criteria being involved, which otherwise may require extra processing.

(6)
figure a
Fig. 3
figure 3

PSO-TPME flowchart

Full size image

3.1 Benchmark Problems

A group of recognized benchmark multi-dimensional functions, which are extensively adopted in the optimization field, were employed to assess the performance of EPSOM, LDW-PSO, PSO-M, M-PSO, GuAPSO and the suggested PSO algorithm in terms of convergence accuracy and convergence speed. It’s worth noting that all PSO variants were tested in the MATLAB environment on an Intel six-core PC with a 2.2 GHz CPU and 16 GB of RAM. The benchmark functions are used herein as minimization problems; each has various characteristics that offer a distinct level of complexity to the tested algorithm. Function characteristics include unimodality or multimodality, symmetric or asymmetric, and separable or inseparable variables. The benchmark multi-dimensional functions are shown in tables 1 and 2 for unimodal (F1-F7) and multimodal (F8-F13) function, respectively. The unimodal functions (F1-F7) are suitable for examining algorithm exploitation since they have just a single global minimum. The Multimodal functions (F8-F13), on the other hand, contain an enormous number of local minima and can asses algorithm’s exploration capabilities [51]. The selected benchmark functions include or exceed the functions that were adopted by the competing PSO variants (Chen (2010) [25], Jiao et al. (2008) [24], Rezaei and Safavi (2020) [34], Wang et al. (2021) [40]).

Rosenbrock function (\(F_5\)) is an unimodal function that is extensively used for local exploration, which was first used in optimization assessment of Genetic Algorithms (GA) by De Jong [52]. According to Shang and Qiu [53], the n-dimensional Rosenbrock function (\(n>4\)) can be a bimodal function, that makes it even a more complex minimization problem, this complexity also originates from the function’s asymmetry and variables inseparability. The global minimum value of Rosenbrock function is zero, which is located at (1, 1, ...).

Rastrigin function (\(F_9\)) is a multimodal function that is employed for the performance assessment of evolutionary algorithms [54]. Although it is extremely multimodal, the positions of the minima are evenly dispersed, the function is symmetric and separable. The global minimum value of Rastrigin function is zero that is located at (0, 0, ...). Griewank function (\(F_{11}\)) is also a multimodal and symmetric function that is widely used for global optimization. The global minimum value of Griewank function is zero that is located at (0, 0, ...). Following Locatelli [55], the function contains a huge number of local minima, which increases exponentially with the number of dimensions. According to Jumonji et al. [54], Griewank function is inseparable in its variables.

Table 1 Unimodal multi-dimensional benchmark functions
Full size table
Table 2 Multimodal multi-dimensional benchmark functions
Full size table

4 Results

Beneficial to evaluate the efficiency of the proposed PSO algorithm on large-scale problems, the benchmark functions (F1-F13) are set to have 30 dimensions. To assess the performance of the proposed PSO algorithm in comparison to EPSOM, LDW-PSO, PSO-M, GuAPSO and M-PSO, for comparison consistency, we have selected identical parameters settings for all the tested PSO variants. The search and initialization spaces are listed in tables 1 and 2 for unimodal (F1-F7) and multimodal (F8-F13) function, respectively. The number of particles is set to 40, and a maximum iteration of 2000. Furthermore, w is declining linearly from \(w_{max} = 0.9\) to \(w_{min} =0.1\), c1 is 1.4962, and c2 is 1.4962. Regarding the M-PSO, the mutation factor mu is set to 0.05.

The proposed PSO method includes four more presetting parameters: one is the classification percentage around the fitness mean, denoted by p, and the other is the elitism with position mutation process initiation iteration, denoted by \(N_e\), the last two parameters are the mutation range and mutation probability, that are denoted by a and mp, respectively. In these simulations, these parameters are selected as \(p=0.02\), \(N_e=3\), \(a=0.5\), and \(mp=1\). Consequently, the particles with fitness in the range of \(\pm 2\%\) of mean particles’ fitness value are classified in the middle category (“fair particles”), the elitism process starts after the third iteration, the mutation range is \(\pm 50\%\) of \(x_{j}(f_{max})\), and the mutation probability is 100%.

Twenty independent simulations were carried out on the previously mentioned functions for each minimization algorithm, which is beneficial to minimizing the statistical errors of the optimization performance of the aforementioned algorithms. The outcomes of each algorithm were averaged across 20 simulations. The averaged fitness performance of PSO variants on 30-dimensional unimodal (F1-F7) benchmark functions is depicted in Fig. 4. The optimization results show that the proposed PSO excels EPSOM, LDW-PSO, PSO-M, GuAPSO and M-PSO in terms of convergence speed and exploitation capabilities, on all tested unimodal benchmark functions. Figure 4h shows a zoomed-in plot of the minimization performance of the proposed PSO on the 30-dimensional unimodal functions. The figure undeniably shows the gigantic improvement of the elitism with targeted position mutation on the proposed PSO after the iteration \(N_e=3\). The bad particles are now exploiting the particle’s position with maximum fitness with high probability of exploring new particles’ positions as a consequence of position mutation.

Fig. 4
figure 4

Fitness performance of PSO variants on multi-dimensional benchmark functions. Figures a–g depict comparison of EPSOM, LDW-PSO, PSO-M, M-PSO, and GuAPSO with the proposed PSO on uni-modal 30-dimensional functions (F1 - F7). Figure h shows a zoomed-in plot of the first 10 iterations performance of the proposed PSO on the F1 - F7

Full size image
Fig. 5
figure 5

Fitness performance of PSO variants on multi-dimensional benchmark functions. Figures a–f depict comparison of EPSOM, LDW-PSO, PSO-M, M-PSO, and GuAPSO with the proposed PSO on multimodal 30-dimensional functions (F8 - F13). Figure g and h shows a zoomed-in plot of the first 10 iterations performance of the proposed PSO on F12 and F13, respectively

Full size image

To assess the global exploration capabilities of the studied optimization methods, the average fitness performance of PSO variants on 30-dimensional multimodal (F8-F13) benchmark functions is presented in Fig. 5. The optimization results for functions F9, F10, and F11 demonstrate that the proposed PSO highly outperforms EPSOM, LDW-PSO, PSO-M, GuAPSO, and M-PSO. This is supported by Fig. 5b, c, and d, in which PSO-TPME converged to global minimum in less than ten iterations, outperforming other evaluated PSO variants by at least two orders of magnitude in terms of convergence speed and accuracy. However, the optimization results for functions F8, F12, and F13 demonstrate that the proposed PSO was not able to converge to the global minimum, as seen in Fig. 5a, e, and f, respectively. The best-performing algorithms in terms of convergence accuracy in functions F8, F12, and F13 were EPSOM, M-PSO, and GuAPSO, respectively, which means each one of them outperforms PSO-TPME in only one function out of 13. On the other hand, PSO-TPME outperforms the other tested PSO variants in ten out of thirteen functions, as follows: three out of six multimodal functions and seven out of seven unimodal functions. Despite the fact that PSO-TPME with 100 % mutation probability has low convergence accuracy in functions F12 and F13, it has high early convergence capabilities in the first ten iterations, as confirmed by Fig. 5g and h. The fast behavior of the proposed PSO and the early exploration capabilities are confirmed for all tested functions, outperforming all tested PSO variants. To investigate why the PSO-TPME with the proposed settings for F12 and F13 was underperforming, functions F12 and F13 were optimized using PSO-TPME with 100% and 1% mutation probability as shown in Fig. 6. The figure clearly shows that the early exploration capabilities are poorer with the 1% mutation probability. The 1% mp case took more than 500 iterations to reach the same performance as the 100% mutation probability in 10 iterations. However, the late exploration capabilities are much better in the 1% mp case, which has superior convergence accuracy. This can be explained by the fact that F12 and F13 are generalized penalized functions with several layers of multitude of local minima. As a result, mutating poor particles with a limited mutation level might result in them staying within one layer of local minima. To avoid falling into this layer, the mutation probability is reduced to allow the poor particle to be updated with the social model, which allows the particles to cross through these layers or at least approach the layer’s boundaries, allowing the poor particles to reach other layers in the case of mutation events.

Fig. 6
figure 6

Fitness performance of PSO-TPME on 30-dimensional functions F12 and F13 with different mutation probability

Full size image

The advantage of elitism-mutation is illustrated in Fig. 7 for function F11. Here, the learning performance of an arbitrary poor particle was tracked for the first ten iterations in the PSO-TPME and PSO w/o elitism-mutation. The classifier assigned the particle as a “poor” after the third iteration. In the fourth iteration, the particle undergoes the elitism routine, which shifts the formerly “poor” particle to the best-performing particle’s position. Subsequently, the particle position is mutated by a 50 % mutation level. This behavior in the PSO-TPME is the driving force behind the excellent exploration and exploitation performance. The fitness of that replaced particle in PSO-TPME is better by more than 15 orders of magnitude as compared to the PSO w/o elitism-mutation. The enhancement in the exploration performance can be seen by the trajectory of the particle in PSO-TPME vs. PSO w/o elitism-mutation.

Fig. 7
figure 7

Learning performance of an arbitrary poor particle while being handled by PSO-TPME and PSO w/o elitism-mutation on 30-dimensional Griewank function (F11) for the first 10 iterations

Full size image
Fig. 8
figure 8

Fitness performance of PSO variants on 30, 60 and 90 dimensional Griewank function (F11) after 10 and 2000 iterations

Full size image
Fig. 9
figure 9

Fitness performance of PSO variants on 30, 60 and 90 dimensional Rastrig function (F9) after 10 and 2000 iterations

Full size image
Fig. 10
figure 10

Fitness performance of PSO variants on 30, 60 and 90 dimensional Rosenbrock function (F5) after 10 and 2000 iterations

Full size image

To assess the effect of the number of dimensions on the performance of the PSO-TPME in terms of convergence speed, early exploration, and late exploration capabilities, the PSO-TPME is tested on 3 selected benchmark functions F5, F9, and F11 with a larger number of dimensions. Figures 8, 9, and 10 show the optimization results for the PSO-TPME, EPSOM, LDW-PSO, PSO-M and M-PSO. The optimal fitness value in the figures is the average optimal one of the solutions in 20 trials. To evaluate the early and late exploration capabilities of the PSO variations, the optimum fitness is estimated after 10 and 2000 iterations, respectively. The optimization was carried out on the previously specified benchmark functions (F5, F9, and F11) with dimensions of 30, 60, and 90 while keeping the same presetting settings for all PSO variations, including the number of particles, constant. The figures undeniably reveal that the PSO-TPME’s early exploration performance outperforms the overall (early and late) exploration performance of the investigated PSO variants by orders of magnitude, based on the functions F5, F9, and F11. This clearly shows that PSO-TPME has remarkably fast and accurate convergence characteristics, which are supported by benchmark functions with various levels of complexity and a large number of dimensions.

Furthermore, the PSO-TPME is evaluated on the same three benchmark functions F5, F9, and F11 with varied mutation probabilities and mutation levels to analyze the influence of mutation probabilities and mutation levels on the performance of the PSO-TPME. Fig. 11a, c, and e show the optimization results for the PSO-TPME with mutation level \(a=0.5\) and varied mutation probabilities (\(mp=\)[1 5 10 20 30 40 50 60 70 80 90 100] %). The figures show that increasing the mutation probability accelerates convergence for all investigated functions. Furthermore, late exploration capabilities like in function F5 can benefit from a relatively low mutation probability. On the other hand, very low mutation probabilities, such as in function F9, might result in immature convergence. The same analysis applies to the mutation level, as seen in Fig. 11b, d, and f, which was tested with \(mp=100\%\) and varied mutation level (\(a=\)[0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1]).

The PSO-TPME with the number of particles \(N=\)[5 10 30 40 60 80 90 100 400], \(mp=100\%\) and \(a=0.5\) is evaluated on the 30-dimensional benchmark functions (F5, F9, and F11) to assess the effect of the number of particles on the performance and the time complexity of the PSO-TPME. As seen in Fig. 12a, c, and e, the convergence accuracy is not affected by the number of particles, whereas, the convergence speed is improved in terms of iterations with a larger number of particles with the penalty of increasing evaluations per iteration. The optimal number of particles for the functions F5, F9, and F11 in terms of the number of evaluations was found to be \(N =\) 10, 30, and 40, respectively. Implying that the optimal number of particles lies in a limited range around the number of dimensions. Fig. 12b, d, and f depict the average calculation time of PSO-TPME for 2000 iterations at various particle numbers. The graphs clearly illustrate that the time complexity of the PSO-TPME increases linearly with particles’ number or in the big \(\mathcal {O}\) notation the time complexity of the PSO-TPME algorithm is \(\mathcal {O}(N)\).

Fig. 11
figure 11

Fitness performance of PSO-TPME on 30-dimensional functions F5, F9 and F11 with \(a=0.5\) and different mutation probabilities (a, c, e), and with \(mp=100\%\) and different mutation levels (b, d, f)

Full size image
Fig. 12
figure 12

Fitness performance (a, c, e) and time complexity (b, d, f) of PSO-TPME on 30-dimensional functions F5, F9 and F11 with different number of particles

Full size image

5 Conclusions

Starting point are five popular variants of particle swarm optimization. The proposed new PSO variant (PSO-TPME) is shown to dramatically improve convergence speed and early exploration capabilities. This variant comprises the three major factors affecting convergence speed and global exploration capabilities: particles’ classification, elitism, and mutation, as well as the original PSO’s cognitive and social models. This variant introduced an alternative classification approach, elitism, and targeted position mutation, all of which were integrated into the basic PSO algorithm. The introduced particle classification process is simple to apply, requires low memory, is adaptive, and provides automated termination criteria based on convergence. These qualities of the proposed classification technique permitted the implementation of targeted elitism and mutation, in terms of targeting just the poor particles, terminating the elitism and mutation process automatically in the event of convergence, and applying different updating models (social and/or cognitive) based on the particle’s category.

A set of benchmark 30-dimensional functions (F1 - F13) widely used in the optimization assessment were used to compare the performance of the proposed PSO-TPME to EPSOM, LDW-PSO, PSO-M, GuAPSO, and M-PSO, in terms of convergence accuracy and convergence speed. The tested functions provide different levels of complexity, such as unimodality or multimodality, symmetry or asymmetry, and separability or inseparability. For each benchmark function, many minimization simulations were performed, repeated, and averaged to reduce statistical errors. The simulations revealed that PSO-TPME surpasses the aforementioned variants by orders of magnitude in terms of convergence rate and accuracy in ten out of the thirteen tested functions and the early exploration capabilities in all of the tested functions.

To assess the effect of the number of dimensions on the performance of the PSO-TPME, the PSO-TPME is tested on three benchmark functions: F5 (Rosenbrock), F9 (Rastrigrin), and F11 (Griewank) with the dimensions of 30, 60, and 90. PSO-TPME maintained the same performance in terms of convergence rate and accuracy, as well as early exploration capabilities, with a larger number of dimensions while keeping the same number of particles. Using the same functions (F5, F9, and F11) with 30 dimensions, a complete sensitivity analysis is performed on PSO-TPME’s key parameters: mutation level, mutation probability, and the number of particles. According to the findings, mutation levels (a) and probabilities (mp) greater than 50% would be an appropriate setting for both faster convergence rates and global exploration capabilities. The optimal number of particles based on the fewest number of evaluations required for convergence falls within a narrow range centered on the number of dimensions.

The authors have already achieved impressive optimization performance in a turbulent jet control experiment in high-dimensional space, featuring many local optima. Fast convergence optimization algorithms such as the proposed PSO are expected to be fruitful for optimization-based control for high-dimensional real-life applications.