Jones N.D.Partial evaluation and automatic program generation.1999
.pdfConversion into continuation passing style 271
An example involving function calls. Consider the problem encountered in Section 5.4.3 where a dynamic function name f was looked up in a static program by the call (lookup f program). Assuming for simplicity that the search is always successful, lookup can be programmed:
(define (lookup f p) (cond
((null? (cdr p)) (take-first-body p)) ((is-first-function? f p) (take-first-body p)) (else (lookup f (remove-first-definition p)))))
Unless we use `the trick', all computations depending on the result returned by the lookup call will be dynamic because n is dynamic. This may be circumvented by introducing a continuation parameter to lookup, yielding lookup1:
(define (lookup1 f p k) (cond
((null? (cdr p)) (k (take-first-body p))) ((is-first-function? f p) (k (take-first-body p))) (else (lookup1 f (remove-first-definition p) k))))
A call (C (lookup f program)) in context C is transformed into (lookup1 f program (lambda (val) (C val))). Again, k gets binding time S ! S, so C is called with a static argument | and the problem is solved by automatic CPS conversion without smart reprogramming.
Functions returning partially static structures. CPS transformation is also very useful for handling functions returning partically static structures; say, a pair (s; d) consisting of a static and dynamic item. We introduce a continuation k de ned by (lambda (s d) . . . ) where s is to be bound to the static component of the answer and d to the dynamic component.
Example 12.1 Suppose an expression e is either the variable x or a sum of two expressions. Then we can write a function eval, which, given an expression and the value of x, returns the size of the expression and its value.
(define (eval e v) (cond
((equal? e 'x) (cons 1 v)) (else
(let ((t1 (eval (cadr e) v)) (t2 (eval (caddr e) v))) (cons (+ 1 (car t1) (car t2)) (+ (cdr t1) (cdr t2)))))))
Here the binding times are mixed and it will not specialize well. In the following example we have introduced a continuation k, which is applied to the size of the expression and its value. To get the ball rolling eval1 must be called with the continuation (lambda (s v) (cons s v)).
272 Binding-Time Improvements
(define (eval1 e v k) (cond
((equal? e 'x) (k 1 v)) ; Apply k to size 1 and value v (else (eval1 (cadr e) ; Evaluate 1st subexpression
v
(lambda (s1 v1) ; Name its two results (eval1 (caddr e) ; Evaluate 2nd subexpression
v
(lambda (s2 v2)
(k (+ 1 s1 s2) (+ v1 v2)))))))))
We assume e static but the value v of x to be dynamic. Then continuation k gets binding time S D ! D, so a call to eval1 with expression (+ x (+ x x)) will specialize to (cons 5 (+ v (+ v v)). Note that the size computation has been
done statically even though intermixed with dynamic operations. |
2 |
Larger-scale applications. J rgensen has used the CPS transformation and selfapplication of Similix to generate a pattern-matching compiler producing better code than that by methods of Peyton Jones [137,138]. Danvy uses CPS to compile non-linear patterns [66]. Another experiment by J rgensen using CPS led to the generation of a compiler for a lazy functional language. The speed of compiled code equalled that of a commercially available compiler [140].
12.3.1Advantages and disadvantages
Transformation into CPS is not always desirable. The resulting programs are of higher-order than the original ones and higher-order facilities are generally harder to manipulate by partial evaluation. A worse problem is that the CPS conversion can a ect termination properties of partial evaluation. A call to the append function (append xs ys) is transformed into (append1 xs ys (lambda (t) t)), where the transformed append1 is de ned by:
(define (append1 xs ys k) (if (null? xs)
(k ys)
(append1 (cdr xs) ys (lambda (t) (k (cons (car xs) t))))))
If xs is dynamic and ys is static then partial evaluation will loop in nitely because the lambdas in the continuation parameter will become more deeply nested for each recursive call, yielding an in nite residual program. This is an accumulating parameter under dynamic control.
Eta conversion 273
12.4Eta conversion
In the lambda calculus community one often uses -conversion: x.Mx , M, if x is not free in M. Although in a certain sense computationally trivial (no operations are performed), it can in some cases improve binding-time separation.
Consider an expression
(let ((f (lambda (x) ...)))
(+ (f ...) (g f)))
where g is dynamic and hence the application (g f) is dynamic. The occurrence of f in the application (g f) is known as a residual code context4 in [28]: f becomes dynamic and the lambda expression also becomes dynamic. Consequently no - reduction of the application (f ...) will take place either. This problem can be overcome by -conversion:
(let ((f (lambda (x) ...)))
(+ (f ...) (g (lambda (z) (f z)))))
As f no longer occurs in a residual code context both (f ...) and (f z) can be-reduced during specialization. Hence the lambda expression (lambda (x) ...) becomes reducible. Note that the new lambda expression inserted by -conversion becomes dynamic.
Eta conversion can also be used for another kind of binding-time improvement. Consider the following expression:
((if (equal? a 2) x y) 3)
Now suppose x and y are higher-order values with x being a static closure and y dynamic. Furthermore suppose that a is static. The conditional will be classi ed as dynamic, because one of the branches is dynamic. This means that we will not be able to reduce the application, not even if a is equal to 2. Using eta conversion the expression can be rewritten into
((if (equal? a 2) x (lambda (z) (y z))) 3)
Now both branches of the condition will be classi ed as static closures, and so will the conditional. Hence we are now able to reduce the application: if a = 2 then specializing the expression will be the result of evaluating (x 3), otherwise if a 6= 2 specializing the expression will result in (y 3).
The two examples demonstrate two di erent kinds of binding-time improvements. In the rst example we had a static closure (f). We encapsulated it with a lambda, which was classi ed as dynamic. In this case eta conversion was used to protect the binding-time of an expression from being classi ed as dynamic. In the second example we had something dynamic (y) in a context where we would like something with the binding-time static closure. In this case eta conversion
4Also known as dynamic code context
274 Binding-Time Improvements
was used to hide the fact that an expression was dynamic. Note that we cannot change the binding time of an expression, which is dynamic for other reasons: y is still dynamic after the transformation.
12.5Improvements derived from `free theorems'
From a function's polymorphic type, a `free theorem' may be derived [222,275]. Holst and Hughes have proposed deriving binding-time improvements from these theorems. Here we show how their technique applies to the standard problem of dynamic lookup in a static list (using ML syntax):
For any natural number i, let select i be the function that selects the ith element of a list. Then select has the type:
8 : N ! List( ) !
Holst and Hughes show that the free theorem for functions of this type is:
f (select i xs) = select i (map f xs)
for any function f : ! . If i is dynamic and xs is static, a binding-time improvement is obtained by replacing the left hand side by the right hand side. The `improved' version appears to be the less e cient, since it applies f to all the elements of the static list, while the original version only applies f to the selected element. A partial evaluator would indeed apply f to all the elements of the static list, but this computation is now entirely static and performed during partial evaluation. The residual program would contain a precalculated list of results f x for all x in xs and a residual selection of the needed result(s).
See papers by Holst and Hughes, and Nielson and Nielson, for more examples and details [119,203,204].
12.6Exercises
Exercise 12.1 Consider the following Scheme expression, where e1 is a static expression and e2 is a dynamic expression.
(if (and e1 e2) e3 e4)
If e1 evaluates to false, the test can be determined statically (at specialization time) since e2 need not be evaluated. Suppose your partial evaluator classi es the entire test (and e1 e2) (and thereby also the conditional) as dynamic. Suggest a transformation that will lead to a better residual program in case e1 evaluates to false. Note that the transformation must not lead to code duplication, and it must leave the strictness properties of the original expression unchanged.
Now consider the following expression:
Exercises 275
(if (or e1 e2) e3 e4)
As above, if e1 evaluates to true, the test can be determined statically. Suppose as above that the entire test will be classi ed as dynamic. Suggest a transformation to solve the problem. Again you must avoid code duplication and leave the strictness
properties unchanged. |
2 |
The following exercises require Similix. Hint: use some of Similix's utility functions to show the binding times of the program, e.g. (showpall) shows all binding times in the current program.
Exercise 12.2 Consider the following Scheme program, where you wish to specialize f with respect to some known list xs and unknown n.
(define (f n xs)
(if (equal? (nth n xs) 42) 'yes 'no))
(define (nth n xs) (if (null? xs)
'error
(if (equal? n 0) (car xs)
(nth (- n 1) (cdr xs)))))
The binding-time analysis performed in Similix will classify the equal? operation in f as dynamic. Because we know (nth n xs) will evaluate to one of the elements of xs, we can apply The Trick to make the equal? operation static. Do this by
introducing a continuation. |
2 |
Exercise 12.3 Consider the following Scheme program where you wish to specialize f with respect to some known x and unknown y. Then z will be marked as dynamic and none of the branches in the if-statement can be reduced.
(define (f x y) (g (h x y)))
(define (h x y) (cons (* x x) y))
(define (g z)
(if (equal? (cdr z) 0) (+ (car z) 2)
(- (car z) 2)))
Introduce a continuation so the branches in the if-statement can be reduced. 2
Exercise 12.4 Extend the nal version of the pattern matcher to handle wildcards in patterns, e.g. * can match any symbol. In what way does this a ect the binding
times? |
2 |
276 Binding-Time Improvements
Exercise 12.5 Consider the following interpreter (written in CPS style) for a version of the lambda calculus. The function run takes a lambda expression E and a value w. The evaluation function ev takes an expression E, an environment env, and a continuation k. An example of a lambda expression could be
(apply (apply (lambda x (lambda y
(+ (var x) (var y)))) (cst 2)) (cst 3))
Identify the place where the continuation k appears in a residual code context and perform an eta conversion (this is the same kind of binding-time improvement as in the rst example in Section 12.4). Identify the place where ev is called with a dynamic continuation and perform an eta conversion (this is the same kind of binding-time improvement as in the second example in Section 12.4).
Try specializing run (with both the initial version and the improved version) with respect to the lambda expression above and w dynamic. The e ect of specializing the improved version should be a conversion of the lambda expression into CPSstyle (the style of the interpreter), besides a translation to Scheme.
;E ::= (cst C) | (var V) | (+ E1 E2)
;| (lambda V E) | (apply E1 E2)
(define (run E w) (ev E (lambda (V) w) (lambda (x) x)))
(define (ev E env k) (cond
((equal? (car E) 'cst) (k (cadr E))) ((equal? (car E) 'var) (k (env (cadr E)))) ((equal? (car E) '+)
(ev (cadr E) env
(lambda (w1) (ev (caddr E)
env
(lambda (w2) (k (+ w1 w2))))))) ((equal? (car E) 'lambda)
(k (lambda (w1 c1) (ev (caddr E)
(lambda (V1)
(if (equal? (cadr E) V1) w1
(env V1))) c1))))
((equal? (car E) 'apply) (ev (cadr E)
env
(lambda (w1)
(ev (caddr E) env (lambda (w2) (w1 w2 k))))))
(else
(error 'ev "unknown syntactic form: ~s" E))))
2
Exercise 12.6 * Discuss the problems involved with automating the process of applying binding-time improvements. Which binding-time improvements can be au-
tomated and which can not? |
2 |
Chapter 13
Applications of Partial
Evaluation
The purpose of this chapter is to show that a wide spectrum of apparently problemspeci c optimizations can be seen as instances of partial evaluation, and that partial evaluation gives a systematic way to devise new optimizations. We shall not, however, attempt a comprehensive coverage of applications to date since the full potential of partial evaluation for solving various practical problems is as yet far from realized, and the distribution of the examples we know of is quite uneven.
In the rst section a selection of problem types that are well suited to optimization by partial evaluation is described, with the hope that the reader may see analogies in his or her own sphere of interest. The second section discusses in more general terms circumstances under which partial evaluation can be of bene t, and points out some pitfalls and ways to overcome them.
Video games. First, an amusing small-scale example of program specialization by hand [224]. The Nevryon game driver needed very fast execution for satisfactory playing speed on an Archimedes personal computer. General code to display a `sprite' in various sizes and motions, and to ip a sprite horizontally and vertically, was found to be too slow. The solution adopted was to write about 20 separate sprite routines, each displaying sprites in slightly di erent ways (moving up, ipped and moving left, etc.). Similar ideas were applied to scrolling and to plotting the backdrop.
13.1Types of problems susceptible to partial evaluation
13.1.1Modularity and related issues
Modular programming is a `good thing' for many reasons. Small modules, each with clearly de ned goals, allow a separation of concerns that increases program portability and eases program reuse and adaptation. This is especially important if
277
278 Applications of Partial Evaluation
programs are frequently changed or replaced, for instance in a scienti c modelling situation where several people or groups, possibly from di erent disciplines, are trying to nd adequate computational models for an external phenomenon.
Modules are often highly parametrized to make them more exible. Similarly, functional programmers use high-level combining constructs such as map, fold, and reduce as `glue' to combine modules in a variety of ways. Modules are often computationally trivial, for example containing only the values of certain key parameters used by other parts of the system, and few or no commands.
There is a cost for a modular, parametrized, high-level programming style: e - ciency. Such programs can spend quite a lot of computation time calling modules, transporting data, perhaps converting data across module interfaces, and creating and invoking closures for functional objects.
Partial evaluation, even with no static data input at all, can speed up such programs. The e ect is to compress modules by merging groups of them together, expanding intermodular calls in place, propagating constants from modules where they are de ned into those where they are used, and precomputing wherever possible. The result is a smaller set of more complex modules, quite likely unsuited for human reading, understanding and modi cation, but signi cantly more e cient1.
13.1.2Parameters with di erent rates of variation
Partial evaluation can help in the following frequently occurring situation:
a function f(x; y) is to be computed for many di erent pairs (x; y),
x is changed less frequently than y, and
a signi cant part of f's computation depends only on x.
Following are a few examples of a non-interpretive nature to illustrate that these conditions are often satis ed; some interpretive examples appear in the next section.
Computer graphics
Mogensen did an experiment at Copenhagen with the `ray-tracing' method of computer graphics [186]. The method is known to give good picture rendition, but is rather slow since it involves tracing the paths of thousands of light rays between various points in a scene to be displayed.
The usual implementation is by a general algorithm which, given a scene and a light ray, performs computations to follow its path. The scene (a collection of 3-dimensional objects) does not change while tracing the light rays, which makes partial evaluation highly relevant.
1One might expect a good compiler to do `constant propagation' at compile time. While true
Types of problems susceptible to partial evaluation 279
Figure 13.1: Computer graphics by ray tracing.
One algorithm has the following overall structure (see Figure 13.1). Variable Point ranges over pixels on a viewing screen, and Viewpoint is the location of the observer's eye. Ray is the line from eye to screen point, extended into the scene being drawn. For any object in the scene, intersect(Object,Ray) nds its intersection point (if any) with the given ray, and variable Intersections is the set of all such intersection points.
The algorithm nds the nearest intersection point Obpoint, i.e. the only point visible to the observer along the current ray, and the object that the ray hits. Finally the light intensity at Obpoint can be calculated using the object's properties such as colour and re ectivity, and this is plotted.
ray-trace(Scene, Screen, Viewpoint): for Point in Screen do
plot(Point, colour(Scene, Viewpoint, Point));
colour(Scene, Viewpoint, Point) =
let Ray = line(Viewpoint, Point) in let Intersections =
fintersect(Object, Ray) j Object in Sceneg in
let (Object, Obpoint) = closest(Viewpoint, Intersections) in shade(Object, Obpoint)
Mogensen optimized this general ray-tracing program by specializing it to a single,xed scene. Concretely, variables Scene and Object were classi ed as static. The
for a single function or procedure, few compilers if any do interprocedural constant propagation, and even fewer do loop unrolling based on the values of constant data.
280 Applications of Partial Evaluation
result was a new program, only good for tracing rays through that particular scene. In experiments the specialized ray tracer was from 8 to 12 times faster than the original. Analysis of the results indicate speedups of about 2.5 from specializing the intersection point computations, and of about 4 from specializing the colour functions.
Sparse systems of linear equations
An early example of problem solving by program generation that amounts to specializing a general program is from 1970 by Gustavson and others [107]. They described a program (called GNSO) which takes as input a sparse system Ax = b of linear equations where A is an N N matrix.
The novelty of their approach is that GNSO generates a program SOLVE from A, which is then executed on input b. It has the form of a long loop-free sequence of Fortran statements. If the input matrix is sparse, the generated program is small and so quite fast. The method works for arbitrary sparsity structures, e.g. it is not restricted to matrices with non-zero elements on a band along the diagonal.
This approach can clearly be thought of as specializing a general Crout algorithm to a particular matrix, and then executing the resulting program. The following is a shortened quote from the article:
In many instances we must solve the system with a xed sparseness structure but with varying numerical values of the elements of A . . . for example (1) linear systems where A depends on parameters which vary from case to case, (2) a system of nonlinear equations by iteration with Newton's method, (3) an initial value problem for partial di erential equations or for a system of ordinary di erential equations by an implicit numerical integration method. . . . The gain is even greater if only b changes from system to system. In this case the factorization of A is done only once, and we just repeat the back substitutions.
Experimental modelling
We believe partial evaluation can substantially increase human and computer e - ciency in experimental modelling over a range of sciences. This section should be taken with a grain of salt, as preliminary studies are under way but results and evaluations have not yet been published.
Computational modelling has some common characteristics:
a natural system is being studied, e.g. oceanographic, meteorological, or ecological processes;
mathematical models are developed to account for evolution of the system's state variables over time and space; and
repeated computer simulation of the model on various initial state conditions is done to compare real-world observations of interesting state variables with the values predicted by the model.