Unicode attempts to provide a consistent way of encoding multilingual text and thus make it possible to exchange text files internationally. The design of Unicode is based on the ASCII code, but goes beyond the Latin alphabet to which ASCII is limited. The Unicode Standard provides the capacity to encode all of the characters used for the written languages of the world. Like ASCII, Unicode assigns each character a unique numeric value and name. Unicode uses three encoding forms that use a common repertoire of characters. These forms allow encoding as many as a million characters.

The three encoding forms of the Unicode Standard allow the same data to be transmitted in a byte, word, or double word format, that is, in 8-, 16or 32-bits per character.

•UTF-8 is a way of transforming all Unicode characters into a variable length encoding of bytes. In this format the Unicode characters corresponding to the familiar ASCII set have the same byte values as ASCII. By the same token, Unicode characters transformed into UTF-8 can be used with existing software.

•UTF-16 is designed to balance efficient access to characters with economical use of storage. It is reasonably compact and all the heavily used characters fit into a single

16-bit code unit, while all other characters are accessible via pairs of 16-bit code units.

•UTF-32 is used where memory space is no concern, but fixed width, single code unit access to characters is desired. In UTF-32 each Unicode character is represented by a single 32-bit code.

3.2 Storage and Encoding of Integers

The Indian mathematician Pingala first described binary numbers in the fifth century B.C. The modern system of binary numbers first appears in the work of Gottfried Leibniz during the seventeenth century. During the mid-nineteenth century the British logician George Boole described a logical system which used binary numbers to represent logical true and false. In 1937, Claude Shannon published his master’s thesis that used Boolean algebra and binary arithmetic to implement electronic relays and switches. The thesis paper entitled A Symbolic Analysis of Relay and Switching Circuits is usually considered to be the origin of modern digital circuit design.

Also in 1937, George Stibitz completed a relay-based computer which could perform binary addition. The Bell Labs Complex Number Computer, also designed by Stibitz, was completed in January 1940. The system was demonstrated to the American Mathematical Society in September 1940. The attendants included John Von Neumann, John Mauchly, and Norbert Wiener. In 1945, von Neumann wrote a seminal paper in which he stated that binary numbers were the ideal computational format.

3.2.1 Signed and Unsigned Representations

For unsigned integers there is little doubt that the binary representation is ideal. Successive bits indicate powers of 2, with the most significant bit at the left and the least significant one on the right, as is customary in decimal representations. Figure 3-1 shows the digit weights and the conventional bit numbering in the binary encoding.

In the low-to-low storage scheme the low-order 8-bits of the operand are stored in the low-order memory byte, the next group of 8-bits are moved to the following memory byte in low-to-high order, and so on. Conceivably, this scheme can be described by saying that the “little end” of the operand is stored first, that is, in lowest memory. According to this notion, the storage scheme is described as the lit- tle-endian format. If the “big-end” of the operand, that is, the highest valued bits, is stored in the low memory addresses then the byte ordering is said to be in big-endian format. Some Intel processors (like those of 80x86 family) follow the lit- tle-endian format. Some Motorola processors (like those of the 68030 family) follow the big-endian format, while others (such as the MIPS 2000) can be configured to store data in either format.

In many situations the programmer needs to be aware of the byte-ordering scheme; for example, to retrieve memory data into processor registers so as to perform multi-byte arithmetic, or to convert data stored in one format to the other one.

This last operation is a simple byte-swap. For example, if the hex value 01020304 is stored in four consecutive memory cells in low-to-high order (little-endian format) it appears in memory (low-to-high) as the values 04030201. Converting this data to the big-endian format consists of swapping the individual bytes so that they are stored in the order 01010304. Figure 3-4 is a diagram of a byte swap operation.

Figure 3-4 Data Format Conversion by Byte Swapping

3.2.4 Sign-Magnitude Representation

Representing signed numbers requires differentiating between positive and negative magnitudes. One possible scheme is to devote one bit to represent the sign. Typically the high-order bit is set (1) to denote negatives and reset (0) to denote positives. Using this convention the decimal numbers 93 and -93 are represented as follows:

01011101 binary = 93 decimal

11011101 binary = -93 decimal

|

|—————————- sign bit

This way of designating negative numbers, called a sign-magnitude representation, corresponds to the conventional way in which we write negative and positive numbers longhand, that is, we precede the number by its sign. Sign-magnitude representation has the following characteristics:

1.The absolute value of positive and negative numbers is the same.

2.Positive numbers can be distinguished from negative numbers by examining the high-order bit.

3.There are two possible representations for zero, one negative (10000000B) and one positive (00000000B).

But a major limitation of sign-magnitude representation is that the processing required to perform addition is different from that for subtraction. Complicated rules are required for the addition of signed numbers. For example, considering two operands labeled x and y, the following rules must be observed for performing signed addition:

1.If x and y have the same sign, they are added directly and the result is given the common sign.

2.If x is larger than y, then y is subtracted from x and the result is given the sign of x.

3.If y is larger than x then x is subtracted from y and the result is given the sign of y.

4.If either x or y is 0 or -0 the result is the non-zero element.

5.If both x and y are -0, then the sum is 0.

However, there are other numeric representations that avoid this situation. A consequence of sign-magnitude representation is that, in some cases, it is necessary to take into account the magnitude of the operands in order to determine the sign of the result. Also, the presence of an encoding for negative zero reduces the numerical range of the representation and is, for most practical uses, an unnecessary complication. An important limitation of using the high-order bit for representing the sign is the resulting halving of the numerical range.

3.2.5 Radix Complement Representation

The radix complement of a number is defined as the difference between the number and the next integer power of the base that is larger than the number. In decimal numbers the radix complement is called the ten’s complement. In the binary system the radix complement is called the two’s complement. For example, the radix complement of the decimal number 89 (ten’s complement) is calculated as follows:

100= higher power of 10

-89

----

11 = ten’s complement of 89

The use of radix complements to simplify machine subtraction operations can best be seen in an example. The operation x = a - b with the following values:

a = 602 b = 353

602

- 353

_____

Note that in the process of performing longhand subtraction we had to perform two borrow operations. Now consider that the radix complement (ten’s complement) of 353 is:

1000 - 353 = 647

Using complements we can reformulate subtraction as the addition of the ten’s complement of the subtrahend, as follows:

602 + 647

______

1249

|____________ discarded digit

The result is adjusted by discarding the digit that overflows the number of digits in the operands.

In performing longhand decimal arithmetic there is little advantage in replacing subtraction with ten’s complement addition. The work of calculating the ten’s complement cancels out any other possible benefit. However, in binary arithmetic the use of radix complements entails significant computational advantages because binary machines can calculate complements efficiently.

The two’s complement of a binary number is obtained in the same manner as the ten’s complement of a decimal number, that is, by subtracting the number from an integer power of the base that is larger than the number. For example, the two’s complement of the binary number 101 is:

While the two’s complement of 10110B is calculated as follows:

You can perform the binary subtraction of 11111B (31 decimal) minus 10110B (22 decimal) by finding the two’s complement of the subtrahend, adding the two operands, and discarding any overflow digit, as follows:

11111B = 31 decimal

+ 01010B = 10 decimal (two’s complement of 22)

_______

101001B discard______|

01001B = 9 decimal (31 minus 22 = 9)

In addition to the radix complement representation, there is a diminished radix representation that is often useful. This encoding, sometimes called the radix-mi- nus-one form, is created by subtracting 1 from an integer power of the base that is larger than the number, then subtracting the operand from this value. In the decimal

system the diminished radix representation is sometimes called the nine’s complement. This is due to the fact that an integer power of ten, minus one, results in one or more 9-digits. In the binary system the diminished radix representation is called the one’s complement. The nine’s complement of the decimal number 76 is calculated as follows:

100 = next highest integer power of 10

99 = 100 minus 1

-76

----

23 = nine’s complement of 89

The one’s complement of a binary number is obtained by subtracting the number from an integer power of the base that is larger than the number, minus one. For example, the one’s complement of the binary number 101 (5 decimal) can be calculated as follows:

An interesting feature of one’s complement is that it can be obtained changing every 1 binary digit to a 0 and every 0 binary digit to a 1. In this example 010B is the one’s complement of 101B. In this context the 0 binary digit is often said to be the complement of the 1 binary digit, and vice versa. Most modern computers contain an instruction that inverts all the digits of a value by changing all 1 digits into 0, and all 0 digits into 1. The operation is also known as logical negation.

Furthermore, the two’s complement can be obtained by adding 1 to the one’s complement of a number. Therefore, instead of calculating

100000B - 10110B

-------

01010B

we can find the two’s complement of 10110B as follows:

10110B = number

01001B = change 0 to 1 and 1 to 0 (one’s complement)

01010B = two’s complement

This algorithm provides a convenient way of calculating the two’s complement in a machine equipped with a complement instruction. Finally, the two’s complement can be obtained by subtracting the operand from zero and discarding the overflow.

The radix complement of a number is the difference between the number and an integer power of the base that is larger than the number. Following this rule, we calculate the radix complement of the binary number 10110 as follows: