Cooper K.Engineering a compiler

correct results.

3.Each instruction contains no more operations of type t than the target machine can issue. This constraint enforces feasibility, since a schedule that

violates it contains instructions that the target machine cannot possibly issue.2

The compiler should only produce schedules that meet all three constraints. Given a well-formed schedule that is both correct and feasible, the length of

the schedule is simply the cycle number in which the last operation completes. This can be computed as

L(S) = max(S(n) + delay(n)).

n N

Assuming that delay captures all of the operational latencies, schedule S should execute in L(S) time.3 With a notion of schedule length comes the notion of a time-optimal schedule. A schedule Si is time-optimal if L(Si ) ≤ L(Sj ), for all other schedules Sj .

The precedence graph captures important properties of the schedule. Computing the total delay along the paths through the graph exposes additional detail about the block. Annotating the precedence graph from Figure 11.1 with information about cumulative latency yields the following graph:

?

i3

The path length from a node to the end of the computation is shown as a superscript on the node. The values clearly show that the path abdfhi is longest—it is the critical path that determines overall execution time.

How, then, should the compiler schedule this computation? An operation can only be scheduled when its operands are available for use. Thus, operations

2Some machines require that each instruction contain precisely the number of operations of type t that the hardware can issue. We think of these machines as vliw computers. This scheme produces a fixed-length instruction with fixed fields and simplifies the implementation of instruction decode and dispatch in the processor. More flexible schemes allow instructions that leave some functional units idle or processing operations issued in earlier cycles.

3Of course, some operations have variable delays. For example, a load operation on a machine with a complex memory hierarchy may have an actual delay that ranges from zero cycles if the requested value is in cache to hundreds or thousands of cycles if the value is in distant memory. If the scheduler assumes the worst case delay, it risks idling the processor for large periods. If it assumes the best case delay, it will stall the processor on a cache miss.

a, c, e, and g are the initial candidates for scheduling. The fact that a lies on the critical path strongly suggests that operation it be scheduled first. Once a has been scheduled, the longest remaining path is cdefhi, suggesting that operation c be scheduled second. With a schedule of ac, b and e tie for the longest path. However, b needs the result of a, which will not be available until the fourth cycle. This makes e followed by b, the better choice. Continuing in this fashion leads to the schedule acebdgfhi. This matches the schedule shown on the left side of Figure 11.1.

However, the compiler cannot simply rearrange the instructions into the proposed order. Notice that operations c and e both define r2. In reordering the code, the scheduler cannot move e before d, unless it renames the result of e to avoid the conflict with the definition of r2 by c. This constraint arises not from the flow of data, as with the dependences modelled in the precedence graph. Instead, it arises from the need to avoid interfering with the dependences modelled in the graph. These constraints are often called anti-dependences.

The scheduler can produce correct code in two ways. It can discover the anti-dependences that exist in the input code and respect them in the final schedule, or it can rename values to avoid them. The example contains four anti-dependences: e to c, e to d, g to e, and g to f. All of them involve the redefinition of r2. (Constraints exist based on r1 as well, but any anti-dependence on r1 duplicates a constraint that also arises from a dependence based on the flow of values.)

Respecting the anti-dependence changes the set of schedules that the compiler can produce. For example, it cannot move e before either c or d. This forces it to produce a schedule such as acbdefghi, which requires eighteen cycles. While this schedule is a ten percent improvement over the unscheduled code (abcdefghi), it is not competitive with the thirty-five percent improvement obtained by renaming to produce acebdgfhi, as shown on the right side of Figure 11.1.

The alternative is to systematically rename the values in the block to eliminate anti-dependences, to schedule, and then to perform register allocation and assignment. This approach frees the scheduler from the constraints imposed by anti-dependences; it creates the potential for problems if the scheduled code requires spill code. The act of renaming, however, does not change the number of live variables; it simply changes their names. It does, however, give the scheduler freedom to move definitions and uses in a way that increases the number of concurrently live values. If the scheduler drives this number too high, the allocator will need to insert spill code—adding long-latency operations and necessitating further scheduling.

The simplest renaming scheme assigns a new name to each value as it is produced. In the ongoing example, this produces the following code:

Digression: How Does Scheduling Relate to Allocation?

The use of specific names can introduce anti-dependences that limit the scheduler’s ability to reorder operations. The scheduler can avoid antidependences by renaming; however, renaming creates a need for the compiler to perform register assignment after scheduling. Instruction scheduling and register allocation interact in complex ways.

The core function of the instruction scheduler is to reorder operations. Since most operations use some values and produce a new value, changing the relative order of two instructions can change the lifetimes of some values. If operation a defines r1 and b defines r2, moving b before a can have several e ects. It lengthens the lifetime of r2 by one cycle, while shortening the lifetime of r1. If one of a’s operands is a last use, then moving b before a lengthens its lifetime. Symmetrically, if one of b’s operands is a last use, the move shortens that operand’s lifetime.

The net e ect of moving b before a depends on the details of a and b, as well as the surrounding code. If none of the uses involved are last uses, then the swap has no net e ect on demand for registers. (Each operation defines a register; swapping them changes the lifetimes of specific registers, but not the aggregate demand.)

In a similar way, register allocation can change the instruction scheduling problem. The core functions of a register allocator are to rename references and to insert memory operations when demand for registers is too high. Both these functions a ect the ability of the scheduler to produce fast code. When the allocator maps a large virtual name space into the name space of target machine registers, it can introduce anti-dependences that constrain the scheduler. Similarly, when the allocator inserts spill code, it adds instructions to the code that must, themselves, be scheduled to hide their latencies.

We know, mathematically, that solving these problems together might produce solutions that cannot be obtained by running the scheduler followed by the allocator, or the allocator followed by the scheduler. However, both problems are complex enough that we will treat them separately. In practice, most compilers treat them separately, as well.

the late 1970s, largely because it discovers reasonable schedules and adapts easily to changes in the underlying computer architectures. However, list scheduling describes an approach rather than a specific algorithm. Wide variation exists in how it is implemented and how it attempts to prioritize instructions for scheduling. This section explores the basic framework of list scheduling, as well as a couple of variations on the idea.

Classic list scheduling operates over a single basic block. Limiting our consideration to straight-line sequences of code allows us to ignore situations that can complicate scheduling. For example, with multiple blocks, an operand might depend on more than one previous definition; this creates uncertainty about when the operand is available for use. With several blocks in its scope, the

scheduler might move an operation from one block to another. This cross-block code motion can force duplication of the operation—moving it into a successor block may create the need for a copy in each successor block. Similarly, it can cause an operation to execute on paths where it did not in the original code— moving it into a predecessor block puts it on the path to each of that block’s successors. Restricting our consideration to the single block case avoids these complications.

To apply list scheduling to a block, the scheduler follows a four step plan:

1.Rename to avoid anti-dependences. To reduce the set of constraints on the scheduler, the compiler renames values. Each definition receives a unique name. This step is not strictly necessary. However, it lets the scheduler find some schedules that the anti-dependences would have prevented. It also simplifies the scheduler’s implementation.

2.Build a precedence graph, P. To build the precedence graph, the scheduler walks the block from bottom to top. For each operation, it constructs a node to represent the newly created value. It adds edges from that node to each node that uses the value. The edges are annotated with the latency of the current operation. (If the scheduler does not perform renaming, P must represent anti-dependence as well.)

3.Assign priorities to each operation. The scheduler will use these priorities to guide it as it picks from the set of available operations at each step. Many priority schemes have been used in list schedulers. The scheduler may compute several di erent scores for each node, using one as the primary ordering and the others to break ties between equally-ranked nodes. The most popular priority scheme uses the length of the longest latencyweighted path from the node to a root of P. We will describe other priority schemes later.

4.Iteratively select an operation and schedule it. The central data structure of the list scheduling algorithm is a list of operations that can legally execute in the current cycle, called the ready list. Every operation on the ready list has the property that its operands are available. The algorithm starts at the first cycle in the block and picks as many operations to issue in that cycle as possible. It then advances the cycle counter and updates the ready list to reflect both the previously issued operations and the passage of time. It repeats this process until every operation has been scheduled.

Renaming and building P are straight forward. The priority computations typically involve a traversal of P. The heart of the algorithm, and the key to understanding it, lies in the final step. Figure 11.3 shows the basic framework for this step, assuming that the target machine has a single functional unit.

The algorithm performs an abstract simulation of the code’s execution. It ignores the details of values and operations to focus on the timing constraints

Cycle ← 1

Ready ← leaves of P

Active ←

while (Ready Active = ) if Ready = then

remove an op from Ready S(op) ← Cycle

Active ← Active op

Cycle ← Cycle + 1

for each op Active

if S(op) + delay(op) ≤ Cycle then remove op from Active

for each successor s of op in P if s is ready then

Ready ← Ready s

Figure 11.3: List Scheduling Algorithm

imposed by edges in P. To accomplish this, it maintains a simulation clock, in the variable Cycle. Cycle is initialized to one, the first operation in the code, and incremented until every operation in the code has been assigned a time to execute.

The algorithm uses two lists to track operations. The first list, called the Ready list, holds any operations that can execute in the current cycle. If an operation is in Ready, all of its operands have been computed. Initially, Ready contains all the leaves of P, since they depend on no other operations. The second list, called the Active list, holds any operations that were issued in an earlier cycle but have not yet completed. At each time step, the scheduler checks Active to find operations that have finished. As each operation finishes, the scheduler checks its successors s in P (the operations that use its result) to determine if all of the operands for s are now available. If they are, it adds s to

Ready.

At each time step, the algorithm follows a simple discipline. It accounts for any operations completed in the previous cycle, then it schedules an operation for the current cycle. The details of the implementation slight obscure this structure because the first time step always has an empty Active list. The while loop begins in the middle of the first time-step. It picks an operation from Ready and schedules it. Next, it increments Cycle to begin the second time step. It updates Active and Ready for the beginning of the new cycle, then wraps around to the top of the loop, where it repeats the process of pick, increment, and update.

The process terminates when every operation has executed to completion. At the end, Cycle contains the simulated running time of the block. If all operations execute in the time specified by delay, and all operands to the leaves

of P are available in the first cycle, this simulated running time should match the actual execution time.

An important question, at this point, is “how good is the schedule that this method generates?” The answer depends, in large part, on how the algorithm picks the operation to remove from the Ready list in each iteration. Consider the simplest scenario, where the Ready list contains at most one item in each iteration. In this restricted case, the algorithm must generate an optimal schedule. Only one operation can execute in the first cycle. (There must be at least one leaf in P, and our restriction ensures that there is at most one.) At each subsequent cycle, the algorithm has no choices to make—either Ready contains an operation and the algorithm schedules it, or Ready is empty and the algorithm schedules nothing to issue in that cycle. The di culty arises when multiple operations are available at some point in the process.

In the ongoing example, P has four leaves: a, c, e, and g. With only one functional unit, the scheduler must select one of the four load operations to execute in the first cycle. This is the role of the priority computation—it assigns each node in P a set of one or more ranks that the scheduler uses to order the nodes for scheduling. The metric suggested earlier, the longest latencyweighted distance to a root in P, corresponds to always choosing the node on the critical path for the current cycle in the current schedule. (Changing earlier choices might well change the current critical path.) To the limited extent that the impact of a scheduling priority is predictable, this scheme should provide balanced pursuit of the longest paths.

E ciency Concerns To pick an operation from the Ready list, as described this far, requires a linear scan over Ready. This makes the cost of creating and maintaining Ready approach O(n2). Replacing the list with a priority queue can reduce the cost of these manipulations to O(n log2 n), for a minor increase in the di culty of implementation. If the

A similar approach can reduce the cost of manipulating the Active list. When the scheduler adds an operation to Active, it can assign it a priority equal to the cycle in which the operation completes. A priority queue that seeks the smallest priority will push all the operations completed in the current cycle to the front, for a small increase in cost over a simple list implementation.

Further improvement is possible with the implementation of Active. The scheduler can maintain a set of separate lists, one for each cycle in which an operation can finish. The number of lists required to cover all the operation latencies is MaxLatency = maxnPdelay(n). When the compiler schedules operation n in Cycle, it adds n to WorkList[(Cycle + delay(n)) mod MaxLatency]. When it goes to update the Ready queue, all of the operations with successors to consider are found in WorkList[Cycle mod MaxLatency]. This schemes uses a small amount of extra space and some extra time to reduce the quadratic cost of searching Active to the linear cost of walking through and emptying the WorkList. (The number of operations in the WorkLists is identical to the number in the Active, so the space overhead is just the cost of having multiple WorkLists. The extra time comes from introducing the subscript calculations on WorkList,

including the mod calculations.)

Caveats: As described, the local list scheduler assumes that all operands for leaves are available on entry to the block. This requires that previous blocks’ schedules include enough slack time at the end for all operations to complete. With more contextual knowledge, the scheduler might prioritize leaves in a way that creates that slack time internally in the block.

The local list scheduler we have described assumes a single functional unit of a single type. Almost no modern computers fit that model. The critical loop of the iteration must select an operation for each functional unit, if possible. If it has several functional units of a given type, it may need to bias its choice for a specific operation by other features of the target machine—for example, if only one of the units can issue memory operations, loads and stores should be scheduled onto that unit first. Similarly, if the register set is partitioned (see Section 10.6.2), the scheduler may need to place an operation on the unit where its operands reside or in a cycle when the inter-partition transfer apparatus is free.

11.3.1Other Priority Schemes

Instruction scheduling, even in the local case, is np-complete for most realistic scenarios. As with many other np-complete problems, greedy heuristic techniques like list scheduling produce fairly good approximations to the optimal solution. However, the behavior of these greedy techniques is rarely robust— small changes in the input may make large di erences in the solution.

One algorithmic way of addressing this instability is careful tie-breaking. When two or more items have the same rank, the implementation chooses among them based on another priority ranking. (In contrast, the max function typically exhibits deterministic behavior—either it retains the first value of maximal rank or the last such value. This introduces a systematic bias toward earlier nodes or later nodes in the list.) A good list scheduler will use several di erent priority rankings to break ties. Among the many priority schemes that have been suggested in the literature are:

•A node’s rank is the total length of the longest path that contains it. This favors, at each step, the critical path in the original code, ignoring intermediate decisions. This tends toward a depth-first traversal of P.

•A node’s rank is the number of immediate successors it has in P. This encourages the scheduler to pursue many distinct paths through the graph— closer to a breadth first approach. It tends to keep more operations on the Ready queue.

•A node’s rank is the total number of descendants it has in P. This amplifies the e ect seen in the previous ranking. Nodes that compute critical values for many other nodes are scheduled early.

•A node’s rank is higher if it has long latency. This tends to schedule the

long latency nodes early in the block, when more operations remain that might be used to cover the latency.

•A node’s rank is higher if it contains the last use of a value. This tends to decrease demand for registers, by moving last uses closer to definitions.

Unfortunately, there is little agreement over either which rankings to use or which order to apply them.

11.3.2Forward versus Backward List Scheduling

An alternate formulation of list schedule works over the precedence graph in the opposite direction, scheduling from roots to leaves. The first operation scheduled executes in the last cycle of the block, and the last operation scheduled executes first. In this form, the algorithm is called backward list scheduling, making the original version forward list scheduling.

A standard practice in the compiler community states is to try several versions of list scheduling on each block and keep the shortest schedule. Typically, the compiler tries both forward and backward list scheduling; it may try more than one priority scheme in each direction. Like many tricks born of experience, this one encodes several important insights. First, list scheduling accounts for a small portion of the compiler’s execution time. The compiler can a ord to try several schemes if it produces better code. Notice that the scheduler can reuse most of the preparatory work—renaming, building P, and some of the computed priorities. Thus, the cost of using multiple schemes amounts to repeating the iterative portion of the scheduler a small number of times. Second, the practice suggests that neither forward scheduling nor backward scheduling always wins. The di erence between forward and backward list scheduling lies in the order in which operations are considered. If the schedule depends critically on the careful ordering of some small set of operations, the two directions may produce radically di erent results. If the critical operations occur near the leaves, forward scheduling seems more likely to consider them together, while backward scheduling must work its way through the remainder of the block to reach them. Symmetrically, if the critical operations occur near the roots, backward scheduling may examine them together while forward scheduling sees them in an order dictated by decisions made at the other end of the block.

To make this latter point more concrete, consider the example shown in Figure 11.4. It shows the precedence graph for a basic block found in the Spec benchmark program go. The compiler added dependences from the store operations to the block-ending branch to ensure that the memory operations complete before the next block begins execution. (Violating this assumption could produce an incorrect value from a subsequent load operation.) Superscripts on nodes in the precedence graph give the latency from the node to the branch; subscripts di erentiate between similar operations. The example assumes operation latencies that appear in the table below the precedence graph.

This example demonstrates the di erence between forward and backward list scheduling. The five store operations take most of the time in the block. The