Cooper K.Engineering a compiler

Figure 11.5: Schedules for the block from go

scheduler.

Why does this happen? The forward scheduler must place all the rank eight operations in the schedule before any rank seven operations. Even though the addI operation is a leaf, its lower rank causes the forward scheduler to defer handling it. By the time that the scheduler runs out of rank eight operations, other rank seven operations are available. In contrast, the backward scheduler places the addI before three of the rank eight operations—a result that the forward scheduler could not consider.

The list scheduling algorithm uses a greedy heuristic to construct a reasonable solution to the problem, but the ranking function does not encode complete knowledge of the problem. To construct a ranking that encoded complete knowledge, the compiler would need to consider all possible schedules—making the ranking process itself np-complete. Thus, an optimal ranking function is impractical, unless p = np.

11.3.3Why Use List Scheduling?

List scheduling has been the dominant algorithm for instruction scheduling for many years. The technique, in its forward and backward forms, uses a greedy heuristic approach to assign each operation an issue slot in the basic block. In practice, it produces excellent results for single blocks.

List scheduling is e cient. It removes an operation from the Ready queue once. It examines each operation for addition to the Ready queue once for each edge that enters its node in P. If an operation has m operands, the scheduler visits its node m times. Each visit looks at each of its m operands, so the amount of work involved in placing it on the ready queue is O(m2). However, for most operations, m is one or two, so this quadratic cost is trivial.

304 CHAPTER 11. INSTRUCTION SCHEDULING

Digression: What About Out-of-Order Execution?

Some processors include hardware support for executing instructions out-of- order (ooo). We refer to such processors as dynamically scheduled machines. This feature is not new; for example, it appeared on the ibm 360/91. To support out-of-order execution, a dynamically-scheduled processor looks ahead in the instruction stream for operations that can execute before they would in a statically-scheduled processor. To do this, the dynamically-scheduled processor builds and maintains a portion of the precedence graph at run-time. It uses this piece of the precedence graph to discover when each operation can execute and issues each operation at the first legal opportunity.

When can an out-of-order processor improve on the static schedule? It cannot avoid waiting for a data dependence, since that represents the actual transmission of a value. It can issue an operation early precisely when the scheduler could have placed the operation earlier—that is, all of the operands are ready before the operation’s appointed cycle. Taken over an entire block, this can snowball—one operation executing early may create opportunities for its descendants in P. The result is a dynamic rewriting of the compilerspecified schedule.

Ooo execution does not eliminate the need for instruction scheduling. Because the lookahead window is finite, bad schedules can defy improvement. For example, a lookahead window of fifty operations will not let the processor execute a string of 100 integer operations followed by 100 floating-point operations in interleaved (integer, floating-point) pairs. It may, however, interleave shorter strings, say of length thirty. Ooo execution helps the compiler by improving good, but non-optimal, schedules.

A related processor feature is dynamic register renaming. This scheme provides the processor with more physical registers than the isa allows the compiler to name. The processor can break anti-dependences that occur within its lookahead window by using physically distinct registers to implement two references connected by an anti-dependence.

List scheduling forms the basis for most algorithms that perform scheduling over regions larger than a single block. Thus, understanding its strengths and its weaknesses is important. Any improvement made to local list scheduling has the potential to improve the regional scheduling algorithms, as well.

11.4Regional Scheduling

As we saw with register allocation, moving from single basic blocks to larger scopes can sometimes improve the quality of code that the compiler generates. With instruction scheduling, many di erent approaches have been proposed for regions larger than a block, but smaller than a whole procedure. This section examines three approaches that derive from list scheduling.

Figure 11.6: Extended basic block scheduling example

11.4.1Scheduling Extended Basic Blocks

An extended basic block (ebb) consists of a series of blocks b1, b2, b3, . . . bn where b1 has multiple predecessors in the control-flow graph and the other blocks bi each have precisely one predecessor, bi−1, 2 ≤ i ≤ n. Extending list scheduling to handle an ebb allows the compiler to consider a longer run of execution during scheduling. Some care must be taken, however, to ensure correct behavior at interim exits from the ebb.

Consider the simple branching structure shown in Figure 11.6. It has several distinct ebbs: (B1, B2, B4), (B1, B3), (B5), and (B6). Each of these ebbs could be subject to ebb scheduling. Notice, however, that the first two ebbs, (B1, B2, B4), (B1, B3) share a common prefix—the block B1. The compiler cannot schedule block B1 in two conflicting ways.

The conflict over B1 can exhibit itself in two ways. The scheduler might move an operation from B1, for example c, past the branch at the end of B1 and down into B2. In this case, c would no longer be on the path from the entry of B1 to the entry of B3, so the scheduler would need to insert a copy of c into B3 to maintain correctness along the path through B3. Alternatively, the scheduler might move an operation from B2, for example f, up into B1. This puts f on the path from the entry of B1 to the entry of B3, where it did not previously execute. This lengthens the path through B3, with the possibility of slowing down execution along that path.

(Without renaming to avoid anti-dependences, it might also change the correctness of that path. To perform renaming in the presence of control flow may require the insertion of copy operations along some of the edges in the controlflow graph in order to maintain correctness. See the discussion of translating from ssa-form back into a linear representation in Section 13.2.)

The compiler has several alternatives for managing these intra-block con-

flicts. To handle the problems that arise from downward motion, it can insert compensation code into blocks that are along exits from the ebb. Any time that it moves an operation from block Bi into one of Bi’s successors, it must consider creating a copy of the instruction in every other successor of Bi. If the value defined by the moved instruction is live in the successor, the instruction is necessary. To handle the problems arising from upward motion, It can restrict the scheduler to prohibit it from moving an operation from its original block into a predecessor block. This avoids lengthening other paths that leave the predecessor block.

(A third choice exists. The compiler might make multiple copies of the block, depending on the surrounding context. In the example, B5 could be cloned to provide two implementations, B5 and B5 , with the edge from B2 connected to B5 and the edge from B3 connected to B5 . This would create longer ebbs containing the new blocks. This kind of transformation is beyond the scope of the current discussion. See Section 14.1.8 for a discussion of cloning.)

To schedule an entire ebb, the compiler performs renaming, if necessary, over the region. Next, it builds a single precedence graph for the entire ebb, ignoring any exits from the ebb. It computes the priority metrics needed to select among ready operations and to break any ties. Finally, it applies the iterative scheduling scheme as with a single block. Each time that it assigns an operation to a specific cycle in the schedule, it must insert any compensation code required by that choice.

The compiler typically schedules each block once. Thus, in our example, the compiler might schedule (B1, B2, B4) first, because it is the longest ebb. Choosing this ebb breaks up the ebb (B1, B3), so only (B3) remains to be scheduled. Thus, the compiler might next schedule (B3), then (B5), and finally, (B6). If the compiler has reason to believe that one ebb executes more often than another, it should give preference to that ebb and schedule it first so that it remains intact for scheduling.

11.4.2Trace Scheduling

Trace scheduling uses information about actual run-time behavior of the program to select regions for scheduling. It uses profile information gathered by running an instrumented version of the program to determine which blocks execute most frequently. From this information, it constructs a trace, or a path through the control-flow graph, representing the most frequently executed path.

Given a trace, the scheduler applies the list scheduling algorithm to the entire trace, in much the same way that ebb scheduling applies it to an ebb. With an arbitrary trace, one additional complication arises—the scheduler may need to insert compensation code at points where control-flow enters the trace (a merge point). If the scheduler has moved an operation upward across the merge point, it needs to copy those operations to the end of the predecessors to the merge that lie outside the trace.

To schedule the entire procedure, the trace scheduler constructs a trace and schedules it. It then removes the blocks in the trace from consideration, and

Figure 11.7: Example loop scheduled for one functional unit

increase is a function of the loop and the number of iterations that the kernel executes concurrently, it is not unusual for the prolog and epilog to double the amount of code required for the pipelined loop.

An example will make these ideas more concrete. Consider the following loop, written in c:

for (i=1; i < 200; i++) z[i] = x[i] * y[i];

Figure 11.7 shows the code that a compiler might generate for this loop, after some optimization. The figure shows the loop scheduled for a single functional unit. The cycle counts, in the first column, have been normalized to the first operation in the loop (at label L1).

To generate this code, the compiler performed operator strength reduction which transformed the address computations into simple additions by the size of the array elements (four in this case). It also replaced the loop-ending test on i with an equivalent test on the address generated for x’s next element, using a transformation called linear function test replacement. The last element of x accessed should be the 199th element, for an o set of (199 − 1) × 4 = 792.

The pre-loop code initializes a pointer for each array and computes an upper bound for the range of r@x into rub. The loop uses rub in cycle six to test for the end of the loop. The loop body loads x and y, performs the multiply, and stores the result into z. It increments the pointers to x, y, and z during the latency of the various memory operations. The comparison fills the cycle after the multiply, and the branch fills the final cycle of the store’s latency.

Figure 11.8 shows the loop scheduled for a machine with two functional units. It assumes a target machine with two functional units, and relies on the strict ordering of iloc operations. All operands are read at the start of the cycle in

Figure 11.8: Example loop scheduled for two functional units

which an operation issues, and all definitions are performed at the end of the cycle in which the operation completes. Loads and stores must execute on unit zero, a typical restriction. The pre-loop code requires two cycles, as compared to four with a single functional unit. The loop body takes eight cycles; when the loop branches back to L1, the store operation is still executing. (On many machines, the hardware would stall the loads until the store completed, adding another cycle to the loop ’s execution time.)

With two functional units, the loop runs in either eight or nine cycles, depending on how the hardware handles the interaction between the store to z and the load from x. The presence of a second functional unit halved the time for the pre-loop code, but did little or nothing for the loop body itself. The scheduled loop still spends a significant portion of its time waiting for loads, stores, and multiplies to finish. This makes it a candidate for a more aggressive the loop scheduling techniques like pipelining.

Figure 11.9 shows the same loop after the compiler has pipelined it. The scheduler folded the loop once, so the kernel works on two distinct iterations concurrently. The prolog section starts up the pipeline by executing the first half of the first iteration. Each iteration of the kernel executes the second half of the ith iteration and the first half of the i + 1st iteration. The kernel terminates after processing the first half of the final iteration, so the epilog finishes the job by performing the second half of the final iteration.

On first examination, the relationship between the code in Figure 11.9 and the original loop is hard to see. Consider the pipelined loop on a line-by-line basis, referring back to Figure 11.8 for reference. All references to specific lines use the cycle number from the leftmost column of Figure 11.9.

The loop prolog To start the loop’s execution, the code must perform all the operations in the pre-loop sequence of the original, along with the first half of iteration number one.

-4: The loadIs are from the pre-loop code. In the non-pipelined loop, they execute in cycle -2 of the pre-loop code.

-3: This is the first instruction of the first iteration. In the non-pipelined loop, it executes as cycle 1 of the loop body.

Figure 11.9: Example loop after pipeline scheduling

-2: This is the second instruction of the first iteration. In the non-pipelined loop, it executes as cycle 2 of the loop body.

-1: These are the other two operations of the pre-loop code. In the non-pipelined loop, they form the instruction at cycle -1.

At the end of the prolog, the code has executed all of the code from the preloop sequence of the original loop, plus the first two instructions from the first iteration. This “fills the pipeline” so that the kernel can begin execution.

The loop kernel Each trip through the loop kernel executes the second half of iteration i, alongside the first half of the i + 1st iteration.

1:In this instruction, the loadAO is from iteration i + 1. It fetches x[i+1]. The addI increments the pointer for x so that it points to x[i+2]. The functional unit reads the value of r@x at the start of the cycle, and stores its new value back at the end of the iteration.

2:In this instruction, the loadAO is from iteration i + 1, while the mult is from iteration i. The mult cannot be scheduled earlier because of the latency of the load for y in the prolog (at cycle -2).

3:In this instruction, the addI is from iteration i+1; it updates the pointer to y. The i2i operation is new. It copies the value of x into another register so that it will not be overwritten before its use in the mult operation during the next kernel operation. (No copy is needed for y because the loadAO for y issues in the same cycle as the mult that uses its old value.

4:In this instruction, the storeAO is from iteration i while the cmp LT is from iteration i + 1. The upper bound on x’s pointer, stored in rub has been

adjusted to terminate the kernel after iteration 198, where the unpipelined loop terminated after iteration 199. The storeAO is scheduled as soon after the corresponding mult as possible, given the latency of the multiply.

5: In this instruction, the conditional branch is from iteration i, as is the addI that increments the pointer to z.

When the kernel finishes execution and branches to L2, all the iterations have executed, except for the second half of the final iteration.

The loop epilog The epilog unwinds the final iteration. Because the scheduler knows that this code implements the last iteration, it can elide the update to r@z and the comparison and branch instructions. (Of course, if the code uses r@z after the loop, the scheduler will discover that it is Live on exit and will include the final increment operation.)

+1: This instruction executes the final multiply operation. Because the scheduler has eliminated the remaining updates and compares, the other operation is a nop.

+3: This instruction executes the final store. Again, the second operation is shown as a nop.

If the scheduler had more context surrounding the loop, it might schedule these epilog operations into the basic block that follows the loop. This would allow productive operations to replace the two nops and to fill cycle +2.

Review In essence, the pipeline scheduler folded the loop in half to reduce the number of cycles spent per iteration. The kernel executes one fewer time than the original loop. Both the prolog and the epilog perform one half of an iteration. The pipelined loop uses many fewer cycles to execute the same operations, by overlapping the multiply and store from iteration i with the loads from iteration i + 1.

In general, the algorithms for producing pipelined loops require analysis techniques that are beyond the scope of this chapter. The curious reader should consult a textbook on advanced optimization techniques for details.

11.5More Aggressive Techniques

Architectural developments in the 1990s increased the importance of instruction scheduling. The widespread use of processors with multiple functional units, increased use of pipelining, and growing memory latencies combined to make realized performance more dependent on execution order. This led to development of more aggressive approaches to scheduling. We will discuss two such approaches.

11.5.1Cloning for Context

In the example from Figure 11.6, two of the ebbs contain more than one block— (B1, B2, B4) and (B1, B3). When the compiler schedules (B1, B2, B4) it must

Figure 11.10: Cloning to Increase Scheduling Context

split (B1, B3) in half. B1 is scheduled into the larger ebb, leaving B3 to be scheduled as a single block. The other two ebbs in the example consist of single blocks as well—(B5) and (B6). Thus, any benefits derived from ebb scheduling occur only on the path B1, B2, B4. Any other path through the fragment encounters code that has been subjected to local list scheduling.

If the compiler chose correctly, and B1, B2, B4 is the most executed path, then splitting (B1, B3) is appropriate. If, instead, another path executes much more frequently, such as (B1, B3, B5, B6), then the benefits of ebb scheduling have been wasted. To increase the number of multiple-block ebbs that can be scheduled, the compiler may clone basic blocks to recapture context. (Cloning also decreases the performance penalty when the compiler’s choice of an execution path is wrong.) With cloning, the compiler replicates blocks that have multiple predecessors to create longer ebbs.

Figure 11.10 shows the result of performing this kind of cloning on the example from Figure 11.6. Block B5 has been cloned to create separate instances for the path from B2 and the path from B3. Similarly, B6 has been cloned twice, to create an instance for each path entering it. This produces a graph with longer ebbs.

If the compiler still believes that (B1, B2, B4) is the hot path, it will schedule (B1, B2, B4, B6) as a single ebb. This leaves two other ebbs to schedule. It will schedule (B5, B6), using the result of scheduling (B1, B2) as context. It will schedule (B3, B5, B6 ), using (B1) as context. Contrast that with the simple ebb scheduler, which scheduled (B3), (B5), and (B6) separately. In that scheme, only (B3) could take advantage of any contextual knowledge about the code that executed before it, since both (B5) and (B6) have multiple predecessors and, therefore, inconsistent context. This extra context cost a second copy of statements j and k and two extra copies of statement l.

Of course, the compiler writer must place some limits on this style of cloning,