304CHAPTER 8. DIAGNOSTICS, FAULT AND CHANGE MANAGEMENT

3.Revise policy for each affected component in the system to reflect the change.

4.Inform users of the impending change and wait for comments.

5.Incorporate any user comments into the policy change.

6.Lock the system to prevent incomplete reconfiguration from being a hazard to the system.

7.Make the changes.

8.Unlock the system.

9.Inform users that the change has been implemented.

Notice that it is important to lock down the system during a major ‘non-atomic’ change. This secures the system against problems that can occur because a part of the system is not yet upgraded. Locking the system prevents policy conflicts from adversely affecting the reliable functioning of the system during the upgrade.

In change management we are interested in consistency, i.e. moving from a state of widespread predictability to a new state of widespread predictability.

8.9 Game-theoretical strategy selection

Game theory is a method of rational decision making. More specifically, it is a method for pitting a set of pre-emptive and defensive strategies against one another, and finding their point of balance in order to maximize gain and minimize loss (see figure 8.12).

Game theory is useful in cases where it is difficult to evaluate the rational gains of following particular policies without some calculational framework. This occurs whenever the number of choices is large and the effects are subtle. Contests which are caused by conflicts of interest between system policy and user wishes, unfold in this framework as environmental interactions which tend to oppose convergence and stability [225, 96]. Game theory is about introducing ‘players’, with goals and aims, into a scheme of rules and then analyzing how much each player can win, according to those restrictions. Each pair of strategies in a game affords the player a characteristic value, often referred to as the ‘payoff’. Game theory has been applied to warfare, to economics (commercial warfare) and many other situations.

Principle 46 (Strategic administration). System administration can be viewed as a strategic game whose aim is to maintain a policy-conformant state [47]. A fault of the system, and its corresponding fault tree, is thus a strategy for driving the system off-policy, while an administrative maintenance strategy is a countermeasure that tends to restore conformance with policy.

Games come in several forms. Some are trivial, one-person games of chance, and are not analyzable in terms of strategies (these are more suitable to ‘flat’ fault tree analysis), since the actions of the players are irrelevant to the outcome;

access to restricted data etc. Such problems are of special interest in connection with security [171, 327].

In a community, games are not necessarily two-player zero-sum engagements. What is lost by one player is not necessarily gained by an obvious opponent. Moreover, the information available to different sides in a conflict can affect their modes of play. In this case the theory of non-zero sum games becomes important; in particular, the idea of a Nash equilibrium arises. The so-called prisoner’s dilemma leads to the famous example of Nash equilibrium [222] which is a trade-off:

Principle 47 (Nash dilemma). A user of the system who pursues solely private interests, does not necessarily promote the best interest of the community as a whole.

Should strategies cooperate or ‘fight’ to maximize their winnings? See the example in section 10.9. The non-zero sum game is beyond the scope of this book, but interested readers are encouraged to find out more about this important method.

Many games can be stated in terms of a basic zero-sum model: it is, for example, the model taken by current system administration agents such as cfengine [41] and PIKT [231], as well as several commercial products, to good effect. Indeed, it is also the view taken by vertebrate immune systems, in detecting potential sickness or damage. Thus, while it might be a simplistic first step where intelligent humans are concerned, it provides a non-trivial example for introductory purposes without overly simplifying the issues. In a realistic situation, both parties in the two-person game would use mixed strategies. A strategy is any specified choice of action. It can involve:

•A schedule of operations,

•A specification of moves and counter-moves (rules).

In addition to simple short-term strategies (tactics), there can be meta-strategies, or long-term goals. For instance, a nominal community strategy might be to implement the stability criteria discussed earlier:

•Maintain the stability of the system

•Maximize total productivity or the generation of work,

•Gain the largest feasible share of resources,

but this might be implemented in the short term by a variety of tactics, such as policy cooperation, non-cooperation and so on. An attack strategy might be to

•Consume or destroy key resources

•Oppose system policy

•Denial of service.

Tactics for attaining intermediate goals might include covert strategies such as bluffing (falsely naming files or other deceptions), taking out an attacker,

counter-attacking, or evasion (concealment), exploitation, trickery, antagonization, incessant complaint (spam), revenge etc. Security and privilege, levels of access, integrity and trust must be woven into algebraic measures for the payoff. Faced with a problem to the system, one may address it either by patching symptoms, or by seeking to root out the fundamental cause. Most successful strategies, including those used by biological life, employ both. A means of expressing all of these devices must be formulated within a model. For an example, see ref. [47].

Figure 8.13: Payoff matrix and a fault tree showing how the fault tree feeds into the game as probabilities, and vice versa. The values in the matrix are probabilistic expressions expressing the likelihood of achieving each strategic goal, weighted by a currency scale for its relative importance. See ref. [48] for details of this game.

The rows and columns of a payoff matrix feed into the lowest level twigs of the fault tree (see figure 8.13). Each row and column represents a pure strategy of the game, but it is known that an optimal mixture of strategies is often the best solution, on average. In a situation where the failure modes are motivated by user actions, not merely random occurrences, this blend of game theory and fault tree analysis has a unique role to play.

Example 8 (Garbage collection). The difficult part of a type II analysis is turning the high-level concepts and aims listed above into precise numerical values. To illustrate the procedure, consider an example of some importance, namely the filling of user disks. The need for forced garbage collection has been argued on several occasions [336, 41, 55], but the effectiveness of different strategies for preventing disks filling may now be analyzed theoretically. This analysis is inspired by the user environment at Oslo University College, and the expressions derived here are designed to model this situation, not an arbitrary system.

The currency of this game must first be agreed upon. What value will be transferred from one player to the other in play? There are three relevant measurements

to take into account: (i) the amount of resources consumed by the attacker (or freed by the defender); sociological rewards: (ii) ‘goodwill’ or (iii) ‘privilege’ which are conferred as a result of sticking to the policy rules. These latter rewards can most easily be combined into an effective variable ‘satisfaction’. A ‘satisfaction’ measure is needed in order to set limits on individuals’ rewards for cheating, or balance the situation in which the system administrator prevents users from using any resources at all. This is clearly not a defensible use of the system, thus the system defences should be penalized for restricting users too much. The characteristic matrix now has two contributions,

Satisfaction πs is assigned arbitrarily on a scale from plus to minus one half, such that,

The different strategies can now be regarded as duels, or games of timing.

These elements of the characteristic matrix must now be filled, using a model and a policy. A general expression for the rate at which users produce files is approximated by:

where rb is the rate at which bad users (i.e. problem users) produce files, and rg is the rate for good users. The total number of users is nu = nb + ng . From experience, the ratio nb/ng is about one percent. The rate can be expressed as a scaled number between zero and one, for convenience, so that rb = 1 − rg .

The payoff in terms of the consumption of resources by users, to the users themselves, can then be modeled as a gradual accumulation of files, in daily waves, which are a maximum around midday:

where the factor of 24 is the human daily rhythm, measured in hours, and Rtot is the total amount of resources to be consumed. Note that, by considering only good users or bad users, one has a corresponding expression for πg and πb, with ru replaced by rg or rb respectively. An automatic garbage collection system (cfengine) results in a negative payoff to users, i.e. a payoff to the system administrator. This may be written

where Tp is the period of execution for the automatic system (in our case, cfengine). This is typically hourly or more often, so the frequency of the automatic cycle is some twenty times greater than that of the human cycle. The rate of resource-freeing ra is also greater than ru, since file deletion takes little time compared with file creation, and also an automated system will be faster than a human. The quota payoff yields a fixed allocation of resources, which are assumed to be distributed equally amongst users and thus each quota slice is assumed to be unavailable to other users. The users are nonchalant, so πs = 0 here, but the quota yields

The matrix elements are expressed in terms of these.

π(1, 1) : Here πs = − 12 since the system administrator is as satisfied as possible by the users’ behavior. πr is the rate of file creation by good users πg , i.e. only legal files are produced. Comparing the strategies, it is clear that

π(1, 1) = π(1, 2) = π(1, 3).

π(1, 4) : Here πs = 0, reflecting the users’ dissatisfaction with the quotas, but the system administrator is penalized for restricting the freedom of the users. With fixed quotas, users cannot generate large temporary files. πq is the fixed quota payoff, a fair slice of the resources. Clearly π(4, 1) = π(4, 2) = π(4, 3) = π(4, 4). The game has a fixed value if this strategy is adopted by the system administrator. However, it does not mean that this is the best strategy, according to the rules of the game, since the system administrator loses points for restrictive practices, which are not in the best interest of the organization. This is yet to be determined.

π(2, 1) : Here πs = 12 since the system administrator is maximally dissatisfied with users’ refusal to tidy their files. The payoff for users is also maximal in taking control of resources, since the system administrator does nothing to prevent this, thus πr = πu. Examining the strategies, one finds that

π(2, 1) = π(3, 1) = π(3, 2) = π(3, 3) = π(4, 1) = π(4, 2).

Figure 8.14: The absolute values of payoff contributions as a function of time (in hours), For daily tidying Tp = 24. User numbers are set in the ratio (ng , nb) = (99, 1), based on rough ratios from the author’s College environment, i.e. one percent of users are considered

mischievous. The filling rates are in the same ratio: rb/Rtot = 0.99, rg /Rtot = 0.01, ra /Rtot = 0.1. The flat dot-slashed line is |πq |, the quota payoff. The lower wavy line is the cumulative

payoff resulting from good users, while the upper line represents the payoff from bad users. The upper line doubles as the magnitude of the payoff |πa | ≥ |πu|, if we apply the restriction that an automatic system can never win back more than users have already taken. Without this restriction, |πa | would be steeper.

As drawn, the daily ripples of the automatic system are in phase with the users’ activity. This is not realistic, since tidying would normally be done at night when user activity is low, however such details need not concern us in this illustrative example.

The policy created in setting up the rules of play for the game penalizes the system administrator for employing strict quotas which restrict users’ activities. Even so, users do not gain much from this, because quotas are constant for all time. A quota is a severe handicap to users in the game, except for very short times before users reach their quota limits. Quotas could be considered cheating by the system administrator, since they determine the final outcome even before play commences. There is no longer an adaptive allocation of resources. Users cannot create temporary files which exceed these hard and fast quotas. An immunity-type model which allows fluctuations is a more resource-efficient strategy in this respect, since it allows users to span all the available resources for short periods of time, without consuming them for ever.

According to the minimax theorem, proved by John von Neumann, any twoperson zero-sum game has a solution, either in terms of a pair of optimal pure strategies or as a pair of optimal mixed strategies [225, 96]. The solution is found as the balance between one player’s attempt to maximize his payoff and the other player’s attempt to minimize the opponent’s result. In general, one can say of the

where the arrows refer to the directions of increasing rows (↓) and columns (→). The left-hand side is the least users can hope to win (or conversely the most that the system administrator can hope to keep) and the right is the most users can hope to win (or conversely the least the system administrator can hope to keep). If we have

it implies the existence of a pair of single, pure strategies (r , c ) which are optimal for both players, regardless of what the other does. If the equality is not satisfied, then the minimax theorem tells us that there exist optimal mixtures of strategies, where each player selects at random from a number of pure strategies with a certain probability weight.

The situation for our time-dependent example matrix is different for small t and for large t. The distinction depends on whether users have had time to exceed fixed quotas or not; thus ‘small t’ refers to times when users are not impeded by the imposition of quotas. For small t, one has:

The ordering of sizes in the above minimum vector is:

This indicates that the equality in eqn. (8.23) is satisfied and there exists at least one pair of pure strategies which is optimal for both players. In this case, the pair is for users to conceal files, regardless of how the system administrator tidies files (the system administrator’s strategies all contribute the same weight in eqn (8.26)). Thus for small times, the users are always winning the game if one assumes that they are allowed to bluff by concealment. If the possibility of concealment or bluffing is removed (perhaps through an improved technology), then the next best strategy is for users to bluff by changing the date, assuming that the tidying looks at the