12-10-2013_09-24-50 / [Simon_Kendal,_Malcolm_Creen]_Introduction_to_Know(BookFi.org)

the system can give an overall diagnosis. However, this input is probably automatic so even though a lot of processing will have to be done by the computer it is not time consuming for the operators.

Answer 2

The system is trying to find out if a fact is true or not, i.e., is the client entitled to housing benefit ‘yes’ or ‘no’.

If backward chaining were used the system would ask only the questions needed to determine the required answer. These facts can be gained while the system is running.

If forward chaining were used all the data would need to be collected up-front. This may require the client to answer many questions that are not relevant to the specified goal. For example, when considering other benefits the system may need to know about the client’s disability status but this may not be relevant when considering the specified goal. Therefore the use of forward chaining could result in many facts being gathered which are never used. Processing an application would become a time consuming and inefficient process.

SECTION 3: DEVELOPING RULE-BASED SYSTEMS

Introduction

In this section you will learn about some of the difficulties of developing rule-based systems and how these can be avoided or overcome.

Objectives

By the end of the section you will be able to:

identify the main problems in developing rule-based systems

evaluate the role of explanation facilities in KBSs

describe the process of evaluating KBSs

describe the process of validating KBSs.

Main Problems in Building a KBS

Two of the main problems to avoid in building a KBS are:

lack of explanation facilities

brittleness.

Explanation Facilities

Explanation facilities are provided within an ES to explain to the user of that system how decisions have been made by the system, i.e., why particular rules have been applied. They are therefore a key part of any ES, with research showing (Wolverton, 1995) that users place significant reliance on explanation facilities.

The need for explanation facilities may appear to be obvious. Humans like to understand why they are being given advice by an expert so that they can see why certain courses of action have been recommended and they can see problems associated with other alternative actions. The output from ESs must therefore provide a similar level of explanation.

However, explanations provided by an ES may not be as detailed as those provided by a human expert. The latter has access to a significant amount of knowledge outside their area of expertise. The ES, on the other hand, only has knowledge in a very specific subject domain. Answers and explanations provided by the system may well be limited because those answers cannot be related to any wider context. Similarly, explanations will be limited where the knowledge base does not provide sufficient detail in the specific subject domain.

Rule Tracing

Rules within a knowledge base can be checked in two ways to ensure that the ES is providing the correct solutions to queries.

A how trace enables the user to find out how an ES has arrived at a conclusion.

A why trace helps the user understand why a particular question is being asked.

While both traces provide the user with some feedback on how the ES is working, they also suffer from the issue of simply providing a chain of reasoning. In other words, the traces simply link a problem and a solution—the ES can explain why a particular conclusion was reached but does not necessarily know why the appropriate rules were in the knowledge base in the first place. The system can state IF something THEN something else happens, but not why those events are linked— providing the ‘why’ sometimes calls for deeper knowledge and understanding.

Building Explanation Text into an ES

Explanation text can be built into the ES to help the user understand the outputs provided. For example, take the situation of a valve regulating water flow illustrated in Figure 4.1.

Input

Liquid

level

Output

FIGURE 4.1. Valve regulating water flow.

When the valve is opened, then water will escape from the system.

A question and answer that can be built into the system could be:

Q.What happens to the water level when the value is opened a little more?

A.It will go down.

To check the logic in this statement, additional explanation can be added:

Q.Why?

A.Because the output flow will be larger than the input flow

Q.Why?

A.Because more water will be released through the output pipe

Q.Why will more water be released through the output pipe?

A.Because opening the valve widens the hole, letting more water through

Q.Why does this happen?. . .

Having placed these explanations into the system, questions can be asked such as:

How did you arrive at that answer?

The ES can then provide the rule explanations to show how the answer was derived.

For example:

User. What happens to the water level when the valve is opened? ES. The water level will go down.

User. How do you know this?

ES. Because I applied Rule 46 which states when the valve opens, more water will be released. I then applied Rule 47 which states when more water is released then the water level will fall.

Activity 11

Here are four rules from an ES that is used to determine whether or not to turn on a heater.

R1: IF door open AND dark THEN ADD cold

R2: IF breezy THEN ADD door open

R3: IF cold THEN ADD start heater AND close door

R4: IF close door THEN DELETE door open

It is currently breezy and dark.

Write out a possible response from the ES in answer to the question.

Why did you apply Rule 2 first?

Feedback 11

The expert system, explaining why it applied Rule 2 first, would have given an explanation similar to the following:

‘I was trying to determine whether or not a heater needs to be turned on. From the information available, I know that it is breezy, so I need to check whether or not a door is open as this is one of the reasons for cold’.

If the system was backward chaining, then it would apply Rule 3 first. If it was forward chaining, then it would apply Rule 2 first simply because it is the only applicable rule.

Dangers of Unintelligent Explanations

Many ES shells support IF. . . . THEN. . . . BECAUSE rules, e.g.

IF you are cold

THEN turn on a heater

BECAUSE this will warm you up

If an ES recommended turning on a heater and the user asked for a justification of this then the ‘BECAUSE’ part of the rule could be displayed to explain the recommendation. However, this is just some text that is regurgitated on demand. There s no ‘intelligence’ behind this justification. In particular there is no mechanism to ensure that this recommendation is appropriate to your individual circumstances. The system would still display the same justification if you were standing in a swimming costume at the North Pole! Clearly in such a situation better advice would be to put on warm clothing.

While explanation texts are useful, there are various dangers to be avoided in writing them.

As systems grow in apparent intelligence they are given more responsibility. Care must be taken not to place too much trust in the ES; the user is still responsible for checking any ES answer for reasonableness.

Adding poor quality or simplistic explanation facilities can inspire undue confidence in a system that does not warrant it. This may mislead the user into taking incorrect decisions.

The apparent intelligence may vastly exceed the true level of understanding. It can be too easy for users to rely on the system believing that it is infallible. Of course, the ES is only as good as its rule base, and if this is incorrect, then solutions from the system will also be wrong. Poor quality explanation facilities may encourage the user to accept erroneous recommendations.

Brittleness

Brittleness is a property displayed by a KBS, where the apparent level of intelligence exceeds the true level of intelligence. The system may appear to be producing appropriate answers however when the problem to be solved requires knowledge not contained within the system it will not be able to solve the problem. Worse still the system may not recognise the limitations of its knowledge and will then propose a faulty solution. This situation can be made worse by the inclusion of an unintelligent explanation facility which will encourage the user to accept the faulty output.

Activity 12

This activity will help you apply your understanding of the concept of brittleness in a KBS to an actual example.

Consider a medical KBS in which brittleness is a characteristic of the knowledge base. What type of problems might emerge in relation to its responses to diagnosis questions?

What might a human doctor do when faced with a failure of their ‘knowledge base’?

Feedback 12

You should have been able to recognise that the dangers associated with brittleness in a medical ES include:

The system will fail to diagnose disorders that were unknown when its knowledge base was developed.

It could reach the wrong diagnosis and try to justify it.

Hopefully, a human doctor will recognise the limits of their own knowledge and will seek help. In other words a human overcomes brittleness by:

remembering previous situations

reasoning from analogous situations

using general knowledge

learning more about the current problem.

It is relatively difficult to build all these features into an ES. Recent work on defining ontologies is helping to overcome the problem of brittleness. By defining an ontology the limits of knowledge contained within a system can be specified and thus it is possible that an ES could also recognise the limitations of its knowledge base (theoretically at least).

We should perhaps note that brittleness does not imply that a system was faulty when developed. An ES may have been able to diagnose every known ailment with 100% accuracy. However, as new ailments and treatments are discovered it is possible that the system will be used to diagnose patients with these disorders and it will then fail to do so as knowledge of these disorders is not included in the knowledge base.

The remainder of this section is devoted to the subject of evaluation and validation of KBSs. It must be stressed at the outset that you do not build a KBS then evaluate and validate it; these activities must be carried out as ongoing parts of the development process itself.

Evaluation of KBS

Evaluation of a KBS is an attempt to assess the overall value of the KBS. Evaluation of a KBS means checking, not only that the KBS has acceptable performance levels, but also that the system is useable, efficient and cost-effective.

The evaluation of a KBS involves two more terms, namely validation and verification.

Validation measures the performance of the KBS. In effect, the output from the system is compared to the output that would be provided by an expert. A check is then made to ensure that the system is performing to an acceptable level of accuracy.

Verification is checking that the KBS has been built correctly, i.e., that the rules are logical and consistent with the knowledge obtained via the knowledge acquisition process.

Evaluation is therefore part of the overall quality control procedures in building a KBS.

The Need for Evaluation

Evaluation of KBS is required in general terms to ensure that knowledge of the real world has been correctly entered into the knowledge base of the KBS.

Activity 13

In this activity you will explore the types of verification available in a number of knowledge engineering tools.

Search the Internet for reference to the tools below and make brief notes about the validation and checking function of any three: