Cooper K.Engineering a compiler

savings may be o set by a larger code size, since each state now includes more code. The new parser, however, has no parse table, performs no table lookups, and lacks the outer loop found in the skeleton parser. While its structure makes it almost unreadable by humans, it should execute more quickly than the tabledriven skeleton parser. With appropriate code layout techniques, the resulting parser can exhibit strong locality in both the instruction cache and the paging system, an arena where seemingly random accesses to large tables produces poor performance.

3.6Practical Issues

3.6.1Handling Context-Sensitive Ambiguity

A second type of ambiguity arises from overloading the meaning of a word in the language. One example of this problem arose in the definitions of Fortran and PL/I. Both these languages use the same characters to enclose the index expression of an array and the argument list of a subroutine or function. Given a textual reference, such as foo(i,j), the compiler cannot tell if foo is a twodimensional array or a procedure that must be invoked. Di erentiating between these two cases requires knowledge of foo’s type. This information is not syntactically obvious. The scanner undoubtedly classifies foo as an Identifier in either case. A function call and an array reference can appear in many of the same situations.

Resolving this ambiguity requires extra-syntactic knowledge. We have used two di erent approaches to solve this problem over the years. First, the scanner can should classify identifiers based on their declared type, rather than their micro-syntactic properties. This requires some hand-shaking between the scanner and the parser; the coordination is not hard to arrange as long as the language has a define-before-use rule. Since the declaration is parsed before the use occurs, the parser can make its internal symbol table available to the scanner to resolve identifiers into distinct classes, like variable-name and function-name. This allows the scanner to return a token type that distinguishes between the function invocation and the array reference.

The alternative is to rewrite the grammar so that it recognizes both the function invocation and the array reference in a single production. In this scheme, the issue is deferred until a later step in translation, when it can be resolved with information from the declarations. The parser must construct a representation that preserves all the information needed by either resolution; the later step will then rewrite the reference into its appropriate form as an array reference or as a function invocation.

3.6.2Optimizing a Grammar

In the parser for a typical Algol-like language, a large share of the time is spent parsing expressions. Consider, again, the classic expression grammar from Section 3.2.3.

Since the parser constructs this sequence in reverse, reading the derivation from bottom line to top line allows us to follow the parser’s actions.

•The left recursive grammar shifts Elt1 onto its stack and immediately reduces it to List. Next, it shifts Elt2 onto the stack and reduces it to List. It proceeds until it has shifted each of the five Eltis onto the stack and reduced them to List. Thus, the stack reaches a maximum depth of

two and an average depth of 106 = 1 23 .

•The right recursive version will shift all five Elti’s onto its stack. Next, it reduces Elt5 to List using rule two, and the remaining Elti’s using rule one. Thus, its maximum stack depth will be five and its average will be

206 = 3 23 .

The right recursive grammar requires more stack space; in fact, its maximum stack depth is bounded only by the length of the list. In contrast, the maximum stack depth of the left recursive grammar is a function of the grammar rather than the input stream.

For short lists, this is not a problem. If, however, the list has hundreds of elements, the di erence in space can be dramatic. If all other issues are equal, the smaller stack height is an advantage.

Associativity Left recursion naturally produces left associativity. Right recursion naturally produces right associativity. In some cases, the order of evaluation makes a di erence. Consider the abstract syntax trees for the five element lists constructed earlier.

The left-recursive grammar reduces Elt1 to a List, then reduces List Elt1, and so on. This produces the ast shown on the left. Similarly, the right recursive grammar produces the ast on the right.

With a list, neither of these orders is obviously correct, although the right recursive ast may seem more natural. Consider, however, the result if we replace the list constructor with addition, as in the grammars

Here, the di erence between the asts for a five-element sum is obvious. The left-recursive grammar generates an ast that implies a left-to-right evaluation order, while the right recursive grammar generates a right-to-left evaluation order.

With some number systems, such as floating-point arithmetic on a computer, this reassociation can produce di erent results. Since floating-point arithmetic actually represents a small mantissa relative to the range of the exponent, addition becomes an identity operation for two numbers that are far enough apart in magnitude. If, for example, the processor’s floating-precision is fourteen decimal digits, and Elt5 − Elt4 > 1015, then the processor will compute Elt5 + Elt4 = Elt5. If the other three values, Elt1, Elt2, and Elt3 are also small relative to Elt5, but their sum is large enough so that

4

Elt5 − Elti > 1014,

i=1

then left-to-right and right-to-left evaluation produce di erent answers.

The issue is not that one evaluation order is inherently correct while the other is wrong. The real issue is that the compiler must preserve the expected evaluation order. If the source language specifies an order for evaluating expressions, the compiler must ensure that the code it generates follows that order. The compiler writer can accomplish this in one of two ways: writing the expression grammar so that it produces the desired order, or taking care to generate the intermediate representation to reflect the opposite associativity, as described in Section 4.4.3.

3.7Summary and Perspective

Almost every compiler contains a parser. For many years, parsing was a subject of intense interest. This led to the development of many di erent techniques for building e cient parsers. The lr(1) family of grammars includes all of the context-free grammars that can be parsed in a deterministic fashion. The tools produce e cient parsers with provably strong error-detection properties. This combination of features, coupled with the widespread availability of parser generators for lr(1), lalr(1), and slr(1) grammars, has decreased interest in other automatic parsing techniques (such as ll(1) and operator precedence).

Top-down, recursive descent parsers have their own set of advantages. They are, arguably, the easiest hand-coded parsers to construct. They provide excellent opportunities for detecting and repairing syntax errors. The compiler writer can more easily finesse ambiguities in the source language that might trouble an lr(1) parser—such as, a language where keyword names can appear as identifiers. They are quite e cient; in fact, a well-constructed top-down, recursive-descent parser can be faster than a table-driven lr(1) parser. (The

direct encoding scheme for lr(1) may overcome this speed advantage.) A compiler writer who wants to construct a hand-coded parser, for whatever reason, is well advised to use the top-down, recursive-descent method.

Questions

1.Consider the task of building a parser for the programming language Scheme. Contrast the e ort required for a top-down, recursive-descent parser with that needed for a table-driven lr(1) parser. (Assume that you have a table-generator handy.)

Now, consider the same contrast for the write statement in Fortran 77. Its syntax is given by the following set of rules.

4.2The Problem

Before it can proceed with translation, the compiler must perform a number of computations that derive information about the code being compiled. For example, the compiler must record the basic information about the type, storage class, and dimension of each variable that its declaration contains, and it must use that information to check the type correctness of the various expressions and statements in the code. It must determine where to insert a conversion between data types, such as from a floating-point number to an integer.

These computations derive their basic facts from information that is implicit in the source program. The compiler uses the results to translate the code into another form. Thus, it is natural for these computations to follow the grammatical structure of the code. This chapter explores two techniques that tie the computation to the grammar used for parsing and use that structure to automate many of the details of the computation.

In Chapter 2, we saw that regular expressions provide most of the features necessary to automate the generation of scanners. The compiler writer specifies a regular expression and a token value for each syntactic category, and the tools produce a scanner. In Chapter 3, we saw that context-free grammars allow automation of much of the work required for parser construction; some of the rough edges must still be smoothed out by a skilled practitioner. The problems of context-sensitive analysis have not yet yielded to a neat, packaged solution.

This chapter presents two approaches to context-sensitive analysis: attribute grammars and ad hoc syntax directed translation.

•Attribute grammars provide a non-procedural formalism for associating computations with a context-free grammar. The definition of the attribute grammar consists of a set of equations; each equation is bound to a specific production. Tools exist for generating automatic evaluators from the set of rules. While attribute grammars have their strong points, they also have weaknesses that have prevented widespread adoption of the formalism for context-sensitive analysis.

•Ad hoc syntax-directed translation requires the compiler writer to produce small small snippets of code, called actions, that perform computations associated with a grammar. The technique draws on some of the insights that underlie attribute grammars to organize the computation, but allows the compiler writer to use arbitrary code in the actions. Despite the lack of formalism, ad hoc syntax directed translation remains the dominant technique used in compilers to perform context-sensitive analysis.

Formalism succeeded in both scanning and parsing, to the point that most compiler writers elect to use scanner generators based on regular expressions and to use parser generators based on context-free grammars. It has not succeeded as well with context-sensitive analysis. By exploring the strengths and weaknesses of attribute grammars, this chapter lays the foundation for understanding the principles that underlie syntax-directed translation. Next, this chapter explores

Figure 4.1: Attribute grammar for signed binary numbers

the mechanisms needed to build translators that perform ad hoc syntax-directed translation and presents a series of examples to illustrate how such computations can be structured. Finally, it concludes with a brief discussion of typical questions that a compiler might try to answer during context-sensitive analysis.

4.3Attribute Grammars

An attributed context-free grammar, or attribute grammar, consists of a contextfree grammar, augmented by a set of rules that specify a computation. Each rule defines one value, or attribute, in terms of the values of other attributes. The rule associates the attribute with a specific grammar symbol; each instance of the grammar symbol that occurs in a parse tree has a corresponding instance of the attribute. Because of the relationship between attribute instances and nodes in the parse tree, implementations are often described as adding fields for the attributes to the nodes of the parse tree.

The simple example shown in Figure 4.1 makes some of these notions concrete. The grammar uses seven productions to describe the language of signed binary numbers. Its grammar has four non-terminals, Number, Sign, List, and Bit, and four terminals +, -, 0, and 1. This particular grammar only associates values with non-terminal symbols. It defines the following attributes:

Number.val, Sign.neg, List.val, List.pos, Bit.val, and Bit.pos. Subscripts are added to grammar symbols when needed to disambiguate a rule; i.e., the occurrences of List in production 5. Notice that values flow from the right-hand side to the left-hand side and vice versa.

Production 4 shows this quite clearly. The pos attribute of bit receives

108 CHAPTER 4. CONTEXT-SENSITIVE ANALYSIS

the value of list.pos. We call bit.pos an inherited attribute because its value is derived from an attribute of its parent (or its siblings) in the parse tree. The val attribute of list receives its value from bit.val. We call list.val a synthesized attribute because its value is derived from an attribute of its children in the tree.

Given a string in the context free grammar, the attribute rules evaluate to set number.val to the decimal value of the binary input string. For example, the string -101 causes the attribution shown on the left side of Figure 4.2. Notice that number.val has the value -5.

To evaluate an instance of the attribute grammar, the attributes specified in the various rules are instantiated for each grammar symbol in the parse (or each node in the parse tree). Thus, each instance of a List node in the example has its own copy of both val and pos. Each rule implicitly defines a set of dependences; the attribute being defined depends on each argument to the rule. Taken over the entire parse tree, these dependences form an attribute dependence graph. Edges in the graph follow the flow of values in the evaluation of a rule; an edge from nodei .fieldj to nodek .fieldl indicates that the rule defining nodek .fieldl uses the value of nodei .fieldj as one of its inputs. The right side of Figure 4.2 shows the dependence graph induced by the parse tree for the string -101.

Any scheme for evaluating attributes must respect the relationships encoded implicitly in the attribute dependence graph. The rule that defines an attribute cannot be evaluated until all of the attribute values that it references have been defined. At that point, the value of the attribute is wholly defined. The attribute is immutable; its value cannot change. This can produce an evaluation order that is wholly unrelated to the order in which the rules appear in the grammar. For a production with three distinct rules, the middle one might evaluate long before the first or third. Evaluation order is determined by the dependences rather than any textual order.

To create and use an attribute grammar, the compiler writer determines a set of attributes for each terminal and non-terminal symbol in the grammar, and designs a set of rules to compute their values. Taken together, these specify a computation. To create an implementation, the compiler writer must create an evaluator; this can be done with an ad hoc program or by using an evaluator generator—the more attractive option. The evaluator generator takes as input the specification for the attribute grammar. It produces the code for an evaluator as its output. This is the attraction of an attribute grammar for the compiler writer; the tools take a high-level, non-procedural specification and automatically produce an implementation.

4.3.1Evaluation Methods

The attribute grammar model has practical use only if we can build evaluators that interpret the rules to automatically evaluate an instance of the problem—a specific parse tree, for example. Many attribute evaluation techniques have been proposed in the literature. In general, they fall into three major categories.

Dynamic Methods These techniques use the structure of a particular instance of the attributed parse tree to determine the evaluation order. Knuth’s original