Balanced Trees

The efficiency of many important operations on trees is related to the Height (= Depth) of the tree - for example searching, inserting, and deleting in a BST are all O(Height).

In general, the relation between Height (H) and the number of nodes (N) in a tree can vary from H = N (degenerate tree) to H = log(N).

For efficiency's sake, we would like to guarantee that H was O(logN). One way to do this is to force our trees to be height-balanced.

A tree is perfectly height-balanced if the left and right subtrees of any node are the same height. e.g.

It is clear that at every level there are twice as many nodes as at the previous level, so we do indeed get H = O(logN). However, perfect height balance is very rare: it is only possible if there are exactly 2^H+1 – 1 nodes!

As a practical alternative, we use trees that are `almost' perfectly height balanced. We will say that a tree is height-balanced if the heights of the left and right subtree's of each node are within 1. The following tree fits this definition:

We will say this tree is height-balanced.

How can we tell if a tree is height-balanced? We have to check every node. The fastest is way is to start at the leaves and work your way up. When you reach a node, you will know the heights of its two subtrees; from that you can tell whether it is height-balanced and also compute the node's height (for use by its parent). For example, in the tree above

C and D are leaves. Their subtrees are all height 0 so C and D are both perfectly balanced. Having finished D we can compute the heights of B's subtrees.

B is not perfectly balanced, but the heights of of its subtrees differ only by 1, so B is regarded as height-balanced. Now we can compute the balance of A.

Like B, A's two subtrees differ by 1 in height. We have now looked at every node; every one is height-balanced, so the tree as a whole is considered to be height-balanced.

What about this tree - is it height-balanced?

Answer: no.

Finally, what about this one?

Answer: no (check node B).

Height-Balanced Binary Search Trees

They are called AVL trees after their Russian inventors Adelson-Velskii and Landis.

It can be proved that in AVL trees: Height < 1.5logN

So if we restrict ourselves to AVL trees the crucial operations of searching, inserting, and deleting are absolutely guaranteed to be O(logN) - providing that height-balance can be maintained in O(logN) time. It can!

In working with AVL trees, operations must preserve two properties:

• (height balanced) heights of left and right subtrees are within 1

• (BST) values in left subtree are smaller than root value, which is smaller than the values in the right subtree.

We will look at the INSERTION operation briefly in this lecture. Deletion follows a similar strategy. For the interested student, a full description of the INSERTION algorithm is given below, after the end of the lecture material.

The INSERTION strategy is this:

1. Add the new value in the tree where it belongs (normal BST insertion).

2. Check if all subtrees are still height-balanced. If they are not, re-balance the tree by changing its shape (i.e. moving around nodes or even whole subtrees).

This is very typical of operations on balanced structures: first you do the operation without regard for the balance, then you check the balance and repair it if necessary.

Let's look at some simple examples. Suppose we start with the empty tree.

I

3

nsert 3:

Insert 6:

3

6

Insert 2:

So far things have been easy! Insert 1:

Insert 0:

The subtrees under `2' are not height balanced. Compute the balance at each node to see this. This is the deepest `unbalanced' node. Fix it.

When we have an imbalance, one of the subtrees is tall or long the other subtree is short. To re-balance, we need to shorten the tall side and/or lengthen the short side. This can be done by what is called a rotation. In our example, `1' rotates up into `2's position, `2' goes down on the opposite side:

Now the left side is 1 shorter and the right side is 1 taller. This re-balanced subtree is joined back to its place in the main tree.

Continuing on, suppose we now insert -1.

Computing the balance at each node, we see that everything is balanced except the top node... so we rotate the tall subtree up and the root down in the opposite direction.

But what do we do with `1's right subtree?

Answer: it becomes the left subtree of the node that rotated down.

Rotation is the operation that is used to restore balance in an AVL tree. For INSERTION, it can be shown that you never need to do more than two rotations (one rotation is not always enough) to restore balance, no matter how large the tree is. For DELETE, rotation is also used to restore balance, but in this case there are circumstances in which fixing the balance in one part of the tree creates an imbalance higher up in the tree. So, in the worst case, you might have to do O(height) rotations during a single DELETE operation.

Topic 13. Sorted list search problem

Given a list A = (a₁, a₂,…, a_i,…, a_n) of distinct items sorted in ascending order and a value k.

It is necessary:

• to find and return index i, such that k = a_i, if k  A;

• to return index -1, if k  A;

The idea of the binary search algorithm:

At the first step it compares k to the middle item in the list A. If k > A[middle] then k is possibly present in the half of the list with larger items. The half of the list with smaller items is excluded from further comparisons.

At the second step k is compared to the middle item in the selected half of the list and so on.

The algorithm stops when:

• It finds A[middle] which is equal to k. A target key is found and middle is returned.

• A current sublist contains only 1 item which is not equal to k. -1 is returned.

This idea is based on the divide and conquer method for algorithm design:

• The original problem for some large input is divided into smaller cases of the same problem.

• The smaller cases are solved separately (often recursively).

• If necessary, the partial solutions are combined to produce the solution for the original input.

Binary search algorithm.

int binary_search_iterative(int A[], int N, int k) // N is array size. n is list size.

{ // Element A[0] is not used. List items A[1],...,A[n] are sorted in ascending order.

// first and last are the indexes of the first and last items of a current sublist.

int n = N-1, first = 1, last = n, middle;

while(first <= last)

{ middle = (first + last)/2; //integer division

if(A[middle] == k)

return middle;

else

if(A[middle] < k) first = middle + 1;

else last = middle - 1;

}//while

return -1; //k is not in the list: A[1],...,A[n];

}//binary_search_iterative

Let us trace how the above algorithm produces its output for the following input:

index: 0 1 2 3 4 5 6 7

int k = 40; int A[8] = { 100, 10, 15, 20, 25, 30, 35, 40}; N = 8; n = 7;

Step#	first	last	first<=last	middle	A[middle] == k	A[middle] < k
Initialize	1	7
Iteration1	1  5	7	1<=7  t	(1+7)/2 =4	25 == 40  f	25<40  t
Iteration2	5  7	7	5<=7  t	(5+7)/2 =6	35 == 40  f	35<40  t
Iteration3	7	7	7<=7  t	(7+7)/2 =7	40 == 40  t	return 7

Note: 1 5 means that first = 1 at the beginning of the 1^st iteration of the loop and first = 5 at the end of this iteration.

Now we assume: int k = 39; int A[8] ={ 100, 10, 15, 20, 25, 30, 35, 40}; N = 8; n = 7;

Step#	first	last	first<=last	middle	A[middle] == k	A[middle] < k
Initialize	1	7
Iteration1	1  5	7	1<=7  t	(1+7)/2 =4	25 == 39  f	25<39  t
Iteration2	5  7	7	5<=7  t	(5+7)/2 =6	35 == 39  f	35<39  t
Iteration3	7	76	7<=7  t	(7+7)/2 =7	40 == 39  f	40<39  f
Iteration4	7	6	7<=6  f			return -1

Note:

• In the loop iteration #4 only while(first <= last) was executed. The body of the loop (the time-consuming instruction) was skipped.

• first > last indicates that the list A does not contain the key k. This situation occurs after unsuccessful search in a single-item sublist (the last hope for success).

W_{Binary_search}(n) = log₂(n) + 1  O(log n);.

90