4.1 The Maze5 Environment

We apply the four algorithms to Maze5 using a population of 1600 classifiers.(n1) Results for the four algorithms are shown in Figure 2. Curves are averaged over ten runs. As Figure 2 shows, XCS evolves a solution for Maze5 that is not optimal (algorithm (i)). Conversely, when generalization does not act, i.e., no #s are used, the system easily reaches the optimum (algorithm (ii)).

When a mechanism to help XCS recover from overly general classifiers is added to XCS, we observe an improvement: both algorithms (iii) and (iv) converge to high performance. Specifically, XCS with biased exploration (algorithm (iv)) slowly converges to a near optimal policy; however, XCSS (algorithm (iii)) rapidly converges to a fully optimal solution that is also stable. The analysis of single runs shows that sometimes XCS with biased exploration fails to converge to a stable solution, while XCSS always reaches the optimum in a stable way. This phenomenon is more evident in the experiments with XCS (algorithm (i)) where in the majority of the cases the system does not reach a stable solution.

Lanzi (1997) observed that XCSS is stable with respect to the population size. To verify this, we applied XCS with biased exploration and XCSS to Maze5 using only 800 classifiers. The results described in Figure 3 show that, even with a small population size, XCSS still converges to a near optimal solution and remains stable. On the contrary, XCS's performance significantly decreases. The analysis of single runs exhibits an increase in the number of experiments in which XCS with biased exploration cannot reach a stable solution leading to a reduction in the overall performance.

4.2 The Maze6 Environment

Maze6 is based on Maze5 but includes a set of obstacles covering a small number of free cells. The two environments are topologically similar, however, the following experiments show that Maze6 is much more difficult for XCS to solve.

In this second experiment, we applied the same four versions of XCS to Maze6. The results described in Figure 4 confirm the results for Maze5. XCS does not converge to an optimal solution when generalization is required while when no # symbols are employed the system easily reaches an optimal performance. Furthermore, there is almost no difference between the performance of XCS with random exploration (i) and XCS with biased exploration (iv). Again, XCS with specify converges to a stable optimum (see Figure 2).

In comparing the performance of XCS in these two environments it is worth noting that, although the two environments are very similar, the performance of XCS in Maze6 is at least five times worse than in Maze5.

These results suggest that when the environment becomes more complex, biased exploration may not guarantee the convergence to a stable solution. Conversely, XCSS evolves a stable near optimal solution for Maze6 even if the population size is reduced to 800 classifiers (see Figure 5).

4.3 The Specify Operator and Biased Exploration

The results presented in this section support the findings previously presented in Lanzi (1997). Specify successfully helps the system recover from situations in which overly general classifiers may corrupt the population before the generalization mechanism of XCS eliminates them. Although biased exploration is adequate in simple environments, such as Maze5, it may become infeasible in more complex environments.

In our opinion, this happens because biased exploration is a global solution to the behavior we discussed, while specify is a local solution. Lanzi (1997) observed that XCS acts in environmental niches and suggested that these should be considered a fundamental element for operators in XCS. Specify follows this principle and directly corrects potentially dangerous situations in the niches where they are detected. Biased exploration on the other hand acts on the whole population and must take into account the structure of the entire environment.