Advanced PHP Programming

18

Profiling

IF YOU PROGRAM PHP PROFESSIONALLY, THERE is little doubt that at some point you will need to improve the performance of an application. If you work on a high-traffic site, this might be a daily or weekly endeavor for you; if your projects are mainly intranet ones, the need may arise less frequently. At some point, though, most applications need to be retuned in order to perform as you want them to.

When I’m giving presentations on performance tuning PHP applications, I like to make the distinction between tuning tools and diagnostic techniques. Until now, this book has largely focused on tuning tools: caching methodologies, system-level tunings, database query optimization, and improved algorithm design. I like to think of these techniques as elements of a toolbox, like a hammer, a torque wrench, or a screwdriver are elements of a handyman’s toolbox. Just as you can’t change a tire with a hammer, you can’t address a database issue by improving a set of regular expressions.Without a good toolset, it’s impossible to fix problems; without the ability to apply the right tool to the job, the tools are equally worthless.

In automobile maintenance, choosing the right tool is a combination of experience and diagnostic insight. Even simple problems benefit from diagnostic techniques. If I have a flat tire, I may be able to patch it, but I need to know where to apply the patch. More complex problems require deeper diagnostics. If my acceleration is sluggish, I could simply guess at the problem and swap out engine parts until performance is acceptable.That method is costly in both time and materials. A much better solution is to run an engine diagnostic test to determine the malfunctioning part.

Software applications are in general much more complex than a car’s engine, yet I often see even experienced developers choosing to make “educated” guesses about the location of performance deficiencies. In spring 2003 the php.net Web sites experienced some extreme slowdowns. Inspection of the Apache Web server logs quickly indicated that the search pages were to blame for the slowdown. However, instead of profiling to find the specific source of the slowdown within those pages, random guessing was used

430 Chapter 18 Profiling

to try to solve the issue.The result was that a problem that should have had a one-hour fix dragged on for days as “solutions” were implemented but did nothing to address the core problem.

Thinking that you can spot the critical inefficiency in a large application by intuition alone is almost always pure hubris. Much as I would not trust a mechanic who claims to know what is wrong with my car without running diagnostic tests or a doctor who claims to know the source of my illness without performing tests, I am inherently skeptical of any programmer who claims to know the source of an application slowdown but does not profile the code.

What Is Needed in a PHP Profiler

A profiler needs to satisfy certain requirements to be acceptable for use:

nTransparency—Enabling the profiler should not require any code change. Having to change your application to accommodate a profiler is both highly inconvenient (and thus prone to being ignored) and intrinsically dishonest because it would by definition alter the control flow of the script.

nMinimal overhead—A profiler needs to impose minimal execution overhead on your scripts. Ideally, the engine should run with no slowdown when a script is not being profiled and almost no slowdown when profiling is enabled. A high overhead means that the profiler cannot be run for production debugging, and it is a large source of internal bias (for example, you need to make sure the profiler is not measuring itself).

nEase of use—This probably goes without saying, but the profiler output needs to be easy to understand. Preferably there should be multiple output formats that you can review offline at your leisure.Tuning often involves a long cycle of introspection and code change. Being able to review old profiles and keep them for later cross-comparison is essential.

A Smorgasbord of Profilers

As with most features of PHP, a few choices are available for script profilers:

nUserspace profilers—An interesting yet fundamentally flawed category of profiler is the userspace profilers.This is a profiler written in PHP.These profilers are interesting because it is always neat to see utilities for working with PHP written in PHP itself. Unfortunately, userspace profilers are heavily flawed because they require code change (every function call to be profiled needs to be modified to hook the profiler calls), and because the profiler code is PHP, there is a heavy bias generated from the profiler running. I can’t recommend userspace profilers for any operations except timing specific functions on a live application where you cannot install an extension-based profiler. Benchmark_Profiler is an example of a

userspace profiler in PEAR, and is available at http://pear.php.net/package/ Benchmark.

n Advanced PHP Debugger (APD)—APD was developed by Daniel Cowgill and me. APD is a PHP extension-based profiler that overrides the execution calls in the Zend Engine to provide high-accuracy timings. Naturally, I am a little biased in its favor, but I think that APD provides the most robust and configurable profiling capabilities of any of the candidates. It creates trace files that are machine readable so they can be postprocessed in a number of different ways. It also provides user-level hooks for output formatting so that you can send profiling results to the browser, to XML, or using any format you wanted. It also provides a stepping, interactive debugger, which us not covered here. APD is available from PEAR’s PECL repository at http://pecl.php.net/apd.

nDBG—DBG is a Zend extension-based debugger and profiler that is available both in a free version and as a commercial product bundled with the commercial PHPEd code editor. DBG has good debugger support but lacks the robust profiling support of APD. DBG is available at http://dd.cron.ru/dbg.

nXdebug—Xdebug is a Zend extension-based profiler debugger written by Derick Rethans. Xdebug is currently the best debugger of the three extension-based solutions, featuring multiple debugger interfaces and a robust feature set. Its profiling capabilities are still behind APD’s, however, especially in the ability to reprocess an existing trace in multiple ways. Xdebug is available from http://xdebug.org.

The rest of this chapter focuses on using APD to profile scripts. If you are attached to another profiler (and by all means, you should always try out all the options), you should be able to apply these lessons to any of the other profilers.The strategies covered here are independent of any particular profiler; only the output examples differ from one profiler to another.

Installing and Using APD

APD is part of PECL and can thus be installed with the PEAR installer:

# pear install apd

After ADP is installed, you should enable it by setting the following in your php.ini file:

zend_extension=/path/to/apd.so

apd.dumpdir=/tmp/traces

APD works by dumping trace files that can be postprocessed with the bundled pprofp trace-processing tool.These traces are dumped into apd.dumpdir, under the name pprof.pid, where pid is the process ID of the process that dumped the trace.

432 Chapter 18 Profiling

To cause a script to be traced, you simply need to call this when you want tracing to start (usually at the top of the script):

apd_set_pprof_trace();

APD works by logging the following events while a script runs:

nWhen a function is entered.

nWhen a function is exited.

nWhen a file is included or required.

Also, whenever a function return is registered, APD checkpoints a set of internal counters and notes how much they have advanced since the previous checkpoint.Three counters are tracked:

nReal Time (a.k.a. wall-clock time)—The actual amount of real time passed.

nUser Time—The amount of time spent executing user code on the CPU.

nSystem Time—The amount of time spent in operating system kernel-level calls.

Accuracy of Internal Timers

APD’s profiling is only as accurate as the systems-level resource measurement tools it has available to it. On FreeBSD, all three of the counters are measured with microsecond accuracy. On Linux (at least as of version 2.4), the User Time and System Time counters are only accurate to the centisecond.

After a trace file has been generated, you analyze it with the pprofp script. pprofp implements a number of sorting and display options that allow you to look at a script’s behavior in a number of different ways through a single trace file. Here is the list of options to pprofp:

Of particular interest are the -t and -T options, which allow you to display a call tree for the script and the entire field of sort options. As indicated, the sort options allow for functions to be sorted either based on the time spent in that function exclusively (that is, not including any time spent in any child function calls) or on time spent, inclusive of function calls.

In general, sorting on real elapsed time (using -r and -R) is most useful because it is the amount of time a visitor to the page actually experiences.This measurement includes time spent idling in database access calls waiting for responses and time spent in any other blocking operations. Although identifying these bottlenecks is useful, you might also want to evaluate the performance of your raw code without counting time spent in input/output (I/O) waiting. For this, the -z and -Z options are useful because they sort only on time spent on the CPU.

A Tracing Example

To see exactly what APD generates, you can run it on the following simple script:

<?php

apd_set_pprof_trace(); hello(“George”); goodbye(“George”);

function hello($name)

{

echo “Hello $name\n”; sleep(1);

}

function goodbye($name)

{

echo “Goodbye $name\n”;

}

?>

Figure 18.1 shows the results of running this profiling with -r.The results are not surprising of course: sleep(1); takes roughly 1 second to complete. (Actually slightly longer than 1 second, this inaccuracy is typical of the sleep function in many languages; you should use usleep() if you need finer-grain accuracy.) hello() and goodbye() are both quite fast. All the functions were executed a single time, and the total script execution time was 1.0214 seconds.

Profiling a Larger Application

Now that you understand the basics of using APD, let’s employ it on a larger project. Serendipity is open-source Web log software written entirely in PHP. Although it is most commonly used for private individuals’Web logs, Serendipity was designed with large, multiuser environments in mind, and it supports an unlimited number of authors.

In this sense, Serendipity is an ideal starting point for a community-based Web site to offer Web logs to its users. As far as features go, Serendipity is ready for that sort of highvolume environment, but the code should first be audited to make sure it will be able to scale well. A profiler is perfect for this sort of analysis.

One of the great things about profiling tools is that they give you easy insight into any code base, even one you might be unfamiliar with. By identifying bottlenecks and pinpointing their locations in code, APD allows you to quickly focus your attention on trouble spots.

A good place to start is profiling the front page of the Web log.To do this, the index.php file is changed to a dump trace. Because the Web log is live, you do not generate a slew of trace files by profiling every page hit, so you can wrap the profile call to make sure it is called only if you manually pass PROFILE=1 on the URL line:

<?php

if($_GET[‘PROFILE’] == 1) { apd_set_pprof_trace();

}

/* ... regular serendipity code starts here ... */

Figure 18.3 shows the profile results for the Serendipity index page, sorted by inclusive real times (using -R). I prefer to start my profiling efforts with -R because it helps give me a good idea which macro-level functions in an application are slow. Because the inclusive timing includes all child calls as well,“top-level” functions tend to be prominent in the listing.

The total time for this page was 0.1231 seconds, which isn’t bad if you are running your own personal site, but it might be too slow if you are trying to implement Serendipity for a large user base or a high-traffic site. include_once() is the top-ranked time-consumer, which is not uncommon in larger applications where a significant portion of the logic is implemented in include files. Note, though, that include_once() not only dominates the inclusive listing, but it seems to dominate the exclusive listing as well. Figure 18.4 verifies this: Rerunning the profile with pprofp -r shows that include_once() takes 29.7% of the runtime, without counting any child function calls.

436 Chapter 18 Profiling

Figure 18.3 Initial profiling results for the Serendipity index page.

Figure 18.4 An exclusive call summary for the Serendipity index page.

What you are seeing here is the cost of compiling all the Serendipity includes. Remember the discussion of compiler caches in Chapter 9,“External Performance Tunings,” that one of the major costs associated with executing PHP scripts is the time spent parsing and compiling them into intermediate code. Because include files are all parsed and compiled at runtime, you can directly see this cost in the example shown in Figure 18.4.You can immediately optimize away this overhead by using a compiler cache. Figure 18.5 shows the effect of installing APC and rerunning the profiles. include_once() is still at the top of inclusive times (which is normal because it includes a large amount of the page logic), but its exclusive time has dropped completely out of the top five calls. Also, script execution time has almost been cut in half.

Figure 18.5 A Serendipity index profile running with an APC compiler cache.

If you look at the calls that remain, you can see that these are the three biggest offenders:

nserendipity_plugin_api::generate_plugins

nserendipity_db_query

nmysql_db_query

You might expect database queries to be slow. Database accesses are commonly the bottleneck in many applications. Spotting and tuning slow SQL queries is covered in Chapter 12,“Interacting with Databases,” so this chapter does not go into detail about that. As predicted earlier, the high real-time cost of the database queries is matched with no user and system time costs because the time that is spent in these queries is exclusively spent on waiting for a response from the database server.

The generate_plugins() function is a different story. Serendipity allows custom user plug-ins for side navigation bar items and comes with a few bundled examples, including a calendar, referrer tracking, and archive search plug-ins. It seems unnecessary for this plug-in generation to be so expensive.

To investigate further, you can generate a complete call tree with this:

> pprofp -tcm /tmp/pprof.28986

Figure 18.6 shows a segment of the call tree that is focused on the beginning of the first call to serendipity_plugin_api::generate_plugins().The first 20 lines or so show what seems to be normal lead-up work. A database query is run (via serendipity_db_query()), and some string formatting is performed. About midway down the page, in the serendipity_drawcalendar() function, the trace starts to look