12-10-2013_09-24-50 / [Simon_Kendal,_Malcolm_Creen]_Introduction_to_Know(BookFi.org)

SECTION 2: CONFIDENCE FACTORS

Introduction

This section shows how confidence factors can be used to manage uncertainty by acting as a measure of it.

Objectives

By the end of the section you will be able to:

use confidence factors in dealing with uncertainty

evaluate the usefulness of confidence factors as a technique for managing uncertainty in knowledge base systems.

Confidence Factors

We need to consider two kinds of uncertainty

Uncertainty in Antecedents

based on the data supplied by the user or

deduced from another rule in the rule base.

Uncertainty in a Rule

based on the expert’s confidence in the rule

based on uncertainty in the data and rules must be combined and propagated to the conclusions.

Imagine we have a rule that states that if A is true, then B is true.

This can be written as:

A => B

If we are uncertain that A is true however, then clearly we are uncertain that B is true. If we are 80% certain that A is true then we will be 80% certain that B is true:

A => B 0.8 0.8

However, in many situations, there is uncertainty concerning the validity of the rule itself. If, given A we are only 80% certain of B we could write this as

0.8

A => B

But what if we are also unsure about A?

0.8

A => B 0.8 ?

In this situation, we can only be 64% certain of event B occurring (0.8 × 0.8 = 0.64).

In other words, if we are only 80% certain that A will occur, we can only be 64% certain of B occurring, i.e., 0.8 × 0.8 = 0.64.

This demonstrates that as we follow an uncertain chain of reasoning, we become less and less certain that the result we obtain is accurate. The way in which confidence is propagated through a chain of reasoning is defined by propagation rules. In this context, ‘rule’ is used in a different sense to the word ‘rule’ in ‘rule based system’.

Reasoning with Confidence Factors

When two independent pieces of corroborating evidence each imply that the result is true, clearly this should make us more certain of the result. If we are 80% certain that A implies C, and 80% certain that B implies C, then if A and B are both true, how confident are we that C is true?

Together, clearly we must be more than 80% confident that C is true, as both of these independent pieces of evidence suggests that is the case; so the answer must be higher than 0.8. But still we cannot be 100% certain, so the answer given must be less than one.

To calculate this we invert the rules; i.e., we take the rule A implies B and say that given A we are 20% (i.e., 100%—80%) certain that B is not true, and given B we are 20% certain that C is not true.

We then multiply these two numbers together (0.2 × 0.2 to give us 0.04) and thus we can say that given A and B, we can be 4% certain that C is NOT true.

Inverting this again (100%—4%) gives us that A and B together means that we are 96% confident that C is true. We therefore get an answer that shows clearly that, given these two corroborating pieces of evidence, we are now very confident, though still not 100% certain, that C is true.

Feedback 1

Our confidence that C is true is 0.8 × 0.8 × 0.6 = 0.38

Activity 2

This activity introduces you to inference networks and provides the opportunity for some practice in calculating combinations of confidence factors.

Look at the following diagram:

1.0

0.9

0.5

This is known as an inference network and illustrates a sequence of relationships between the facts A to E and the confidence factors assigned to the facts as well as the rules connecting them.

The first three equations governing the application of different confident factors and their combination as we work through the network from left to right are as follows:

There are two separate rules being applied here to the ways in which confidence factors are combined, depending on the context. What do you think these rules are?

What justification is there for the difference in these rules?

Complete the calculation for the fourth equation:

CF(E) = CF(B&D) × CF (IF B&D THEN E) =

Feedback 2

You should have been able to interpret the rules being applied in the equations as follows:

The first equation applies the rule whereby the confidence factors for the fact A and for the rule IF A THEN B are multiplied together to give the confidence for the fact B.

The second equation applies the same rule to calculate the CF(D).

This is appropriate since we begin with certainty (CF = 1) for fact A and this is adjusted for the 0.8 confidence in the rule itself IF A THEN B. Similarly, we begin with CF = 0.5 for fact C and then adjust for the confidence in the rule IF C THEN D.

The third equation adopts a different approach and takes the minimum of CF(B) and CF(D) since the lack of confidence in the combination of the two components can only be as high as the confidence in the weakest (link) of the two.

The fourth equation can be completed as follows:

CF(E) = CF(B&D) × CF(IF B&D THEN E) = 0.25 × 0.9 = 0.225

Strengths of Confidence Factors

The main strengths of using confidence factors is that they allow us to:

express varying degrees of confidence, which allows these values to be manipulated

rank several possible solutions, especially if not too much emphasis is placed on the actual numerical values generated.

It is in this latter respect particularly that confidence factors differ from probabilities, which are calculated values.

Limitations of Confidence Factors

The limitations of confidence factors include:

Confidence factors are generated from the opinions of one or more experts, and thus there is very little hard evidence for these numbers in practise. People are notoriously unreliable when assigning numbers to express levels of confidence.

As well as two people finding very different numbers, individuals will also be inconsistent on a day-to-day basis on placing values on confidence factors.

Not withstanding the comment above if a doctor said they were:

90% confident that a patient had pneumonia

5% confident that a patient had the flu

1% confident that a patient had a common cold.

Without placing too much emphasis on the actual numbers we can see that the doctor strongly believes that the patient has pneumonia, and while we recognise that other possibilities exist, the patient should receive the appropriate care for this.

Summary

In this section you have learned about confidence factors and how these can be applied to rules in a knowledge base in order to allow meaning to be extracted from the knowledge even where uncertainty exists.

SECTION 3: PROBABILISTIC REASONING

Introduction

This section introduces the principle of probabilistic reasoning, and shows how Bayes theorem can be used to determine the extent of that uncertainty, firstly in a written example, and then using formulae.

Objectives

By the end of the section you will be able to:

explain probabilistic reasoning and how to define probabilities within knowledge based systems

explain and use Bayes theorem.

Bayesian Inference

Probability theory originated with Pascal in the seventeenth century. In the eighteenth century Reverend Bayes developed a theorem that forms the basis of conditional probability. Most attempts to use probability theory to handle uncertainty in KBS are based on Bayes theorem.

If enough data, of the right sort, is available, statistical analysis based on conditional probability theory is the best way to handle uncertainty. However, there is often simply not enough data to produce adequate sample sizes, or the data does not have the necessary properties, such as independence from one another.

Bayes theorem can be represented by the following equation.

P(A|B) = P(B|A)P(A)

P(B)

In other words, the probability (P) of some event A occurring given that event B has occurred is equal to the probability of event B occurring given that event A has occurred, multiplied by the probability of event A occurring and divided by the probability of event B occurring.

A Bayesian inference system can be established using the following steps.

1.Define a set of hypotheses, which define the actual results expected.

2.Assign a probability factor to each hypothesis to give an initial assessment of the likelihood of that outcome occurring.

set of possible hypothesis is complete. Each hypothesis is given a probability of occurring, providing a guide to how often that outcome can be expected.

For example, the set of outcomes from throwing a dice can include the hypothesis that an even number is thrown (50% probability) or an odd number is thrown (also a 50% probability).

Similarly, a set of hypotheses can be produced for the different diseases that a person could be suffering from. Probabilities can be calculated for each disease showing how likely it is that the patient has that disease.

Checking the Evidence Produced

The accuracy of the probabilities attached to each hypothesis will be tested by collecting evidence about the outcome actually achieved. In effect, the hypothesis is proved by ensuring that the evidence actually falls within one of the expected hypotheses.

Amending the Probabilities

The idea the probabilities must be assigned to each hypothesis introduces one of the main points of Bayesian inference: some assumption must be made concerning the initial probabilities of each hypothesis occurring.

However, as evidence is obtained showing whether or not each outcome was determined correctly from the facts available, these probabilities can be updated to provide a better match to reality. In turn, this enables the expert system to provide more accurate answers to the problems presented to it.

Example of the Application of Bayes theorem

The hypothesis (H) is that a patient visiting a doctor has the flu. The events (E) are the symptoms that are presented by that patient such as:

running nose

sneezing

high temperature

headache.

The prior probability based on previous experience is that P(flu) = 0.3, or there is a 30% chance that any patient walking into the doctors surgery has the flu. This probability will be amended as information about the patient becomes known.

In this case let’s imagine that the patient does have a high temperature, runny nose and is sneezing, but has no headache. How do we determine the specific probability of flu given this particular set of symptoms?

This data is quite easy to obtain, and given Bayes theorem, we can then calculate the probability of the hypothesis; i.e., that someone has the flu given any combination of symptoms we choose. In doing this we have changed our data collection slightly but significantly.

Instead of fixing the symptom, and then determining how many patients with this symptom have the flu, we first sample 100 people with the flu and determine the probabilities of the symptoms appearing. Similarly, we sample 100 people who do not have the flu, and determine the probabilities of the same symptoms appearing.

Imagine that we wanted to determine the probability of the hypothesis if two symptoms were evident but a third symptom was not, e.g.

P(flu: given high temperature and runny nose but no headache) = ???

We cannot measure: P(H:E1 and E2 and not E3) as this would require finding 100 previous patients with this exact combination of symptoms. At the same time we would need 100 patients with every other possible set of symptoms—and there could be an almost infinite set of combinations.

However, we can measure the probability where three different events will occur to support the hypothesis:

P(E1 : H), P(E2 : H), P(E3 : H)

Also, we can measure

P(E1 : not H), P(E2 : not H), P(E3 : not H)

i.e., testing that the three different events do not support the hypothesis.

Using Bayes theorem we can then calculate the probability of H given E1 and E2 occurring to support the hypothesis but E3 not occurring:

P(H : E1 and E2 and not E3)

Assume that the following data is available.

In this case, we sample 100 people who have the flu and determine that 70% of them have a high temperature.

In 100 people who do not have the flu, 50% of these have a high temperature.