9.2 The Role of Subsumption Deletion

We now address how the representation of the task that the agent is learning can be compacted in environments, such as Maze6, where a pressure toward more general classifiers cannot be developed.

Subsumption deletion was introduced by Wilson (1997) to improve generalization with XCS. However, early experiments with Maze5, Maze6 and Woods 14 (not reported here) show that its introduction may decrease system performance and, in some cases, prevent the system from converging to a stable policy.

Next, we analyze an important aspect of subsumption deletion in order to explain why it may compromise XCS's performance and how the observed behavior is related to specify and teletransportation.

Subsumption deletion acts when new classifiers created by the GA must be inserted in the population and replaces offspring classifiers with clones of their parents if: (1) the offspring classifiers are more specific than their parents, and (2) the parameters of their parents have been updated sufficiently. As has been observed (Lanzi, 1997a), XCS with subsumption deletion evolves formal generalizations: #s are not inserted in the classifiers because they are necessary in order to match more niches; instead, the system converges to a population in which classifiers contain as many # symbols as possible without becoming inaccurate. XCS with subsumption deletion tends to produce classifiers which apply to many more conditions than those the agent experienced. They are also likely to be inaccurate if the environment is extended -for example, if a new area of the environment is discovered.

In such cases the specify operator can be useful in recovering from classifiers that are overly general in the new area of the environment. In fact, all the classifiers that are overly general in the new area will become inaccurate and specify will be activated.

This aspect of subsumption deletion is strictly related to the accuracy parameter and therefore to teletransportation. In fact, subsumption deletion relies upon the accuracy parameter and thus it may corrupt the population when overly general classifiers are evaluated as accurate. We developed a series of experiments in which XCS with biased exploration was applied to the environments previously presented. Results (not reported here) show that the performance of the system is highly decreased when subsumption deletion is used.

After we introduced teletransportation, we repeated the set of experiments to test whether the decrease of performance when subsumption was used depended on the presence of overly general classifiers that were evaluated as accurate.

As Figure 15 shows, XCST's performance is still optimal when subsumption deletion is employed. The comparison of the population size in macroclassifiers for the two systems in Figure 16 shows that it compacts the representation producing a smaller population. These results suggest that the decrease in XCS's performance when XCS employs subsumption deletion is related to the problem of local learning that was introduced in the previous sections.