Embedded Robotics (Thomas Braunl, 2 ed, 2006)

22 Behavior-Based Systems

result in a faster completion time, up to the point where too many robots encumber each other (e.g. stopping to avoid collisions or accidentally destroying each other’s cluster), resulting in an increasing completion time [Du, Bräunl 2003].

Figure 22.3: Cube clustering with real robots and in simulation

22.4 Behavior Framework

The objective of a behavior framework is to simplify the design and implementation of behavior-based programs for a robot platform such as the EyeBot. At its foundation is a programming interface for consistently specified behaviors.

We adapt the convention of referring to simple behaviors as schemas and extend the term to encompass any processing element of a control system. The specification of these schemas is made at an abstract level so that they may be generically manipulated by higher-level logic and/or other schemas without specific knowledge of implementation details.

Schemas may be recursively combined either by programming or by generation from a user interface. Aggregating different schemas together enables more sophisticated behaviors to be produced. The mechanism of arbitration between grouped schemas is up to the system designer. When combined with coordination schemas to select between available behaviors, the outputs of the contributing modules can be directed to actuator schemas to produce actual robot actions. A commonly used technique is to use a weighted sum of all schemas that drive an actuator as the final control signal.

Behavior design The framework architecture was inspired by AuRA’s reactive component [Arkin, Balch 1997], and takes implementation cues from the TeamBots environment realization [Balch 2006].

The basic unit of the framework is a schema, which may be perceptual (for example a sensor reading) or behavioral (for example move to a location). A schema is defined as a unit that produces an output of a pre-defined type. In

330

Behavior Framework

our implementation, the simplest types emitted by schemas are integer, floating point, and boolean scalar values. More complex types that have been implemented are the two-dimensional floating point vector and image types. The floating point vector may be used to encode any two-dimensional quantity commonly used by robot schemas, such as velocities and positions. The image type corresponds to the image structure used by the RoBIOS image processing routines.

Schemas may optionally embed other schemas for use as inputs. Data of the pre-defined primitive types is exchanged between schemas. In this way behaviors may be recursively combined to produce more complex behaviors.

In a robot control program, schema organization is represented by a processing tree. Sensors form the leaf nodes, implemented as embedded schemas. The complexity of the behaviors that embed sensors varies, from simple movement in a fixed direction to ball detection using an image processing algorithm. The output of the tree’s root node is used every processing cycle to determine the robot’s next action. Usually the root node corresponds to an actuator output value. In this case output from the root node directly produces robot action.

Behavior The behavioral framework has been implemented in C++, using the implementation RoBIOS API to interface with the Eyebot. These same functions are simulated and available in EyeSim (see Chapter 13), enabling programs created with the

framework to be used on both the real and simulated platforms.

The framework has been implemented with an object-oriented methodology. There is a parent Node class that is directly inherited by type-emitting schema classes for each pre-defined type. For example, the NodeInt class represents a node that emits an integer output. Every schema inherits from a node child class, and is thus a type of node itself.

All schema classes define a value(t) function that returns a primitive type value at a given time t. The return type of this function is dependent on the class – for example, schemas deriving from NodeInt return an integer type. Embedding of schemas is by a recursive calling structure through the schema tree. Each schema class that can embed nodes keeps an array of pointers to the embedded instances. When a schema requires an embedded node value, it iterates through the array and calls each embedded schema’s respective value(t) function. This organization allows invalid connections between schemas to be detected at compile time: when a schema embedding a node of an invalid type tries to call the value function, the returned value will not be of the required type. The compiler checks that the types from connected emitting and embedding nodes are the same at compilation time and will flag any mismatch to the programmer.

The hierarchy of schema connections forms a tree, with actuators and sensors mediated by various schemas and schema aggregations. Time has been discretized into units, Schemas in the tree are evaluated from the lowest level (sensors) to the highest from a master clock value generated by the running program.

331

22 Behavior-Based Systems

Schemas A small working set of schemas using the framework was created for use in a neural network controller design task. The set of schemas with a short description of each is listed in Table 22.1. The schemas shown are either perceptual (for example Camera, PSD), behavioral (for example Avoid), or generic (for example Fixed vector). Perceptual schemas only emit a value of some type that is used by the behavioral schemas. Behavioral schemas transform their input into an egocentric output vector that would fulfill its goal.

A front-end program has been created to allow point-and-click assemblage of a new schema from pre-programmed modules [Venkitachalam 2002]. The representation of the control program as a tree of schemas maps directly to the interface presented to the user (Figure 22.4).

For a schema to be recognized by the user interface, the programmer must “tag” the header file with a description of the module. A sample description block is shown in Program 22.1. The graphical user interface then parses the header files of a schema source directory to determine how to present the modules to the user.

Table 22.1: Behavior schemas

The header block specifies how a particular schema interconnects with other schemas. It includes a description of typed initialization parameters for the module, a list of ports that can be used for emitting or embedding other modules, and various meta-information.

From the interconnection of the visual modules, the user interface generates appropriate code to represent the tree specified by the user. The structure of the program is determined by analyzing the behavior tree and translating it into a series of instantiations and embedding calls. The uniform nature of the behavioral API facilitates a simple code generation algorithm.

332

22 Behavior-Based Systems

arcs. Evolution of the structure to achieve an objective task is performed by the genetic algorithm. The set of parameters describing the neural network arc weights is optimized to produce a controller capable of performing the task.

Our implementation of a genetic algorithm uses a direct binary encoding scheme to encode the numeric weights and optionally the thresholds of the controller’s neural network. A single, complete, neural network controller configuration is encoded into a chromosome. The chromosome itself is a concatenation of individual floating point genes. Each gene encodes a single weight of the neural network. The population consists of a number of chromosomes, initially evaluated for fitness and then evolved through numerous iterations. Population evolution is achieved by a single point crossover operation at gene boundaries on a set percentage of the population. This is supplemented by mutation operations (random bit-wise inversion) on a small percentage of the population, set as a user parameter. The top performing members of a population are preserved between iterations of the algorithm (elitism). The lowest performing are removed and replaced by copies of the highest performing chromosomes. In our trials we have used population sizes of between 100 and 250 chromosomes.

The simulator environment was built around an early version of EyeSim 5 [Waggershauser 2002]. EyeSim is a sophisticated multi-agent simulation of the Eyebot hardware platform set in a virtual 3D environment. As well as simulating standard motor and hardware sensors, the environment model allows realistic simulation of image capture by an on-board camera sensor (Figure 22.5). This allows for complete testing and debugging of programs and behaviors using image processing routines.

Figure 22.5: EyeSim 5 simulator screen shot

Because we run our programs in a simulated environment, we can obtain records of the positions and orientations of all objects in the environment with perfect accuracy. The logging calls determine positions during execution from

334

Adaptive Controller

the simulator’s internal world model. The control program of the robot calls these functions after it completes execution and writes them in a suitable file format for reading by the separate evolutionary algorithm. The results are not used by the robot to enhance its performance while running. The final output of the program to the logfile is analyzed after termination to determine how well the robot performed its task.

Program 22.2: Schema example definition

1#include "Vec2.h"

2#include "NodeVec2.h"

3class v_Random : public NodeVec2

4{ public:

5v_Random(int seed);

6~v_Random();

7Vec2* value(long timestamp);

9private:

10Vec2* vector;

Program 22.3: Schema example implementation

3{ double phi, r;

4srand( (unsigned int) seed);

5phi = 2*M_PI / (rand() % 360);

6r = (rand() % 1000) / 1000.0;

7vector = new Vec2(phi, r);

8lastTimestamp = -1;

9}

10

11v_Random::~v_Random()

12{ if(vector)

13delete vector;

14}

15

16Vec2* v_Random::value(long timestamp)

17{ if(timestamp > lastTimestamp)

18{ lastTimestamp = timestamp;

19// Generate a new random vector for this timestamp

20vector->setx(phi = 2*M_PI / (rand() % 360));

21vector->sety(r = (rand() % 1000) / 1000.0);

22}

23return vector;

24}

335

22 Behavior-Based Systems

Program 22.4: Avoid schema

5this->maxspeed = maxspeed;

6}

7Vec2* v_Avoid_iii::value(long timestamp)

8{ double front, left, right;

9if(timestamp != lastTimestamp) {

10// Get PSD readings

11frontPSD = (NodeInt*) embeddedNodes[0];

12leftPSD = (NodeInt*) embeddedNodes[1];

13rightPSD = (NodeInt*) embeddedNodes[2];

14front = frontPSD->value(timestamp);

15left = leftPSD->value(timestamp);

16right = rightPSD->value(timestamp);

17// Calculate avoidance vector

18// Ignore object if out of range

19if (front >= sense_range) front = sense_range;

20if (left >= sense_range) left = sense_range;

21if (right >= sense_range) right = sense_range;

22...

23

Schemas that embed a node (i.e. take the output of another node as input) must allocate space for these nodes in their constructor. A method to do this is already available in the base class (initEmbeddedNodes), so the schema only needs to specify how many nodes to allocate. For example, the avoid schema

336

Tracking Problem

embeds three integer schemas; hence the constructor calls initEmbeddedNodes shown in Program 22.4. The embedded nodes are then accessible in an array embeddedNodes. By casting these to their known base classes and calling their value methods, their outputs can be read and processed by the embedding schema.

22.6 Tracking Problem

The “avoid detected obstacles” schema embeds PSD sensor schemas as inputs, mounted on the front, left, and right of the robot (Figure 22.6). These readings are used to determine a vector away from any close obstacle (see Figure 22.6). Activation of the “avoid detected obstacles” schema prevents collisions with walls or other objects, and getting stuck in areas with a clear exit.

Figure 22.6: Avoidance schema

337

22 Behavior-Based Systems

Ball detection is achieved by a hue recognition algorithm that processes images captured from the Eyebot camera (Figure 22.5) and returns ball position in the x-direction and ball height as “high-level sensor signals”. The system should learn to activate the “turn left” behavior whenever the ball drifts toward the left image border and the “turn right” behavior whenever the balls drifts to the right. If the sensors detect the ball roughly in the middle, the system should learn to activate the “drive straight” behavior.

At the moment, only one behavior can be active at a time. However, as a future extension, one could combine multiple active behaviors by calculating a weighted sum of their respective vector outputs.

22.7 Neural Network Controller

The role of the neural network controller is to select the currently active behavior from the primitive schemas during each processing cycle. The active behavior will then take control over the robot’s actuators and drive the robot in the direction it desires. In principle, the neural network receives information from all sensor inputs, status inputs from all schemas, and a clock value to determine the activity of each of the schemas. Inputs may be in the form of raw sensor readings or processed sensor results such as distances, positions, and pre-processed image data. Information is processed through a number of hidden layers and fed into the output layer. The controller’s output neurons are responsible for selecting the active schema.

An additional output neuron is used to have the controller learn when it has finished the given task (here driving close to the ball and then stop). If the controller does not stop at a maximal number of time steps, it will be terminated and the last state is analyzed for calculating the fitness function. Using these fitness values, the parameters of the neural network controller are evolved by the genetic algorithm as described in Chapter 20.

We decided to use an off-line learning approach for the following reasons:

•Generation of ideal behavior

There is the possibility of the system adapting to a state that fulfills some but not all of the task’s fitness criteria. This typically happens when the method of learning relies on gradient descent and becomes stuck in a local fitness maxima [Gurney 2002]. Off-line evolution allows the application of more complex (and hence processor intensive) optimization techniques to avoid this situation.

•Time to convergence

The time a robot takes to converge to an appropriate controller state reduces the robot’s effective working time. By evolving suitable parameters off-line, the robot is in a suitable working state at run-time.

338

Neural Network Controller

•System testing

Evolution of behavior in a real environment limits our ability to test the controller’s suitability for a task. Off-line evolution enables extensive testing of the system in simulation before actual use.

•Avoid physical damage to system

While the controller is evolving, its response may cause damage to the physical robot until it learns to perform a task safely. Evolving the controller in simulation allows such responses to be modified before real harm can be done to expensive hardware.

ball size

turn left turn right

ball x-pos.

drive straight

Figure 22.7: Neural network structure used

Figure 22.7 shows the neural network structure used. Image processing is done in an “intelligent sensor”, so the ball position and ball size in the image are determined by image processing for all frames and directly fed into the network’s input layer. The output layer has three nodes, which determine the robot’s action, either turning left, turning right, or driving forward. The neuron with the highest output value is selected in every cycle.

Each chromosome holds an array of floating point numbers representing the weights of the neural network arbitrator. The numbers are mapped first-to-last in the neural network as is demonstrated for a simpler example network in Figure 22.8.

Figure 22.8: Chromosome encoding

339