CHAPTER 23 ■ OPTIMIZING DRUPAL

# Attempt to serve from cache return(lookup);

}

The above VCL snippet checks if the request is from a logged-in user (one that has a cookie starting with "SESS") and if it not then normalizes the Cookie header by removing it altogether. If there is a need to have some cookies from anonymous request echoed to your backend servers then you can adjust the Cookie regex or add a few more lines to be more selective about which cookies ought to miss the Varnish cache lookup pahse.

The other incoming request header that needs to be normalized is Accept-Encoding because it varies slightly across different web browser types. The most common use of the Accept-Encoding header if for the web browser to communicate to the web server that the browser can receive compressed content. The typical VCL snippet to normalize the Accept-Encoding looks like this:

# Normalize Accept-Encoding to get better cache coherency if (req.http.Accept-Encoding) {

#No point in compressing media that is already compressed if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {

remove req.http.Accept-Encoding;

#MSIE 6 JS bug workaround

}elsif(req.http.User-Agent ~ "MSIE 6") { unset req.http.Accept-Encoding;

}elsif (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip";

}elsif (req.http.Accept-Encoding ~ "deflate") { set req.http.Accept-Encoding = "deflate";

}else {

# unkown algorithm

remove req.http.Accept-Encoding;

}

}

Varnish: finding extraneous cookies

The following command line on your Varnish server is useful for watching live incoming Cookie headers that being echoed from Varnish to your backend servers.

varnishlog | grep TxHeader | grep Cookie

This is useful for adjusting how the Cookie header is filtered in Varnish.

Boost

The popular Boost module for Drupal (http://drupal.org/project/boost) essentially builds a static file cache for dynamically generated Drupal content. With the Boost module installed in Drupal, whenever Drupal generates a dynamic page, Boost will save a static copy of that content so that the next anonymous request for that same page will be delivered from the Boost cache. A background cron process periodically culls outdated pages from the Boost cache, which are then regenerated on the next

508

CHAPTER 23 ■ OPTIMIZING DRUPAL

request. This approach reduces overall PHP and MySQL overhead but still requires Apache (or Nginx, IIS, lighthttpd) to process a few extra rewrite rules for each page request.

The key to good Boost performance is to put the Boost cache directory on a fast local file system. Some Drupal admins may consider writing Boost cache files into a shared network file system so that many web servers can share the same Boost cache files; however, a busy web site can have a lot of file system I/O arise from Boost cache maintenance, so much so that a network shared file system slows down considerably, in which case the Boost cache ought to be a local directory on each web server instead.

If each web server has extra memory but slow disks, then you may also consider writing your Boost cache files to a local ramfs file system, which is a feature of Linux that allows you to create an ephemeral storage volume that exists entirely in RAM.

Boost vs. Varnish

Although Boost and Varnish are different kinds of caching solutions, Drupal administrators often weigh these two options directly against each other. In general Boost is easier to set up and administer than Varnish. However, Varnish offers a general solution to better performance as it can be used to proxy cache other kinds of content, such as static images and style sheets, and not just Drupal pages. Varnish also offers the ability to load balance and rewrite requests before they even reach your web server, whereas Boost requests are still hitting the web server.

However, it’s also possible to use Boost and Varnish together. You may just need to tune your HTTP cache expiration headers and Boost cache purging so that Varnish and Boost are refreshing their caches in a timely manner.

Linux System Tuning for High Traffic Servers

Tuning Linux to handle high volumes of web traffic deserves a book unto itself. There are, however, simple changes that will help improve the performance of high traffic sites, such as those outlined in the sysctl_set.sh script here (courtesy of Audun Ytterdal, http://www.varnish-cache.org/lists/pipermail/ varnish-misc/2008-April/001763.html).

#!/bin/sh

# Tweaks (see http://varnish-cache.org/wiki/Performance) echo "

net.ipv4.ip_local_port_range = 1024 65536 net.ipv4.tcp_rmem=4096 87380 16777216

net.ipv4.tcp_wmem=4096 65536 16777216 net.ipv4.tcp_fin_timeout = 3 net.ipv4.tcp_no_metrics_save=1 net.ipv4.tcp_syncookies = 0 net.ipv4.tcp_max_orphans = 262144 net.ipv4.tcp_max_syn_backlog = 262144 net.ipv4.tcp_synack_retries = 2

net.ipv4.tcp_syn_retries = 2 net.core.rmem_max=16777216

509

510

CHAPTER 23 ■ OPTIMIZING DRUPAL

Examples of slow file systems include (compared to the foregoing choices):

virtual disks (inside any virtualized server environment)

NFS and other types of software-driven network file shares

software raided disks (depending on the chosen raid level)

S3FS (mounts Amazon S3 storage as a local disk)

LVM (slows down as more volume snapshots are retained)

Much of LAMP stack design involves deciding on which volumes to store web content and database tables based on the size, speed, and reliability of the file system. Your best performance choice is to use the fastest file system available and ensure uptime and integrity with redundancy (ie., multiple redundant servers and database replication).

Dedicated Servers vs. Virtual Servers

Dedicated physical servers are going to outperform virtual servers when it comes to network I/O, disk I/O, and memory I/O, even in situations where the virtual server supposedly has been allocated more resources (CPU, disk, and memory) than a dedicated server of similar specs. An important factor to consider is that in a virtualized server environment, the CPU, disk I/O, memory I/O, and network I/O have added I/O routing layers between the server OS and the actual hardware. And, therefore, all I/O operations are subject to task scheduling whims of the host hypervisor as well as the demands of neighboring virtual machines on the same physical host.

As a real example, a virtual server hosting a database server may have twice as much CPU power as a cheaper physical dedicated server; however, the virtual server may also have an added 1ms network latency (a very real example from an actual Xen virtualized environment), even between neighboring virtual machines. Now, 1ms network latency doesn’t seem like enough latency to care about, until you consider that a logged-in Drupal page request may involve hundreds of serialized MySQL queries; thus the total network latency overhead can amount to a full second of your page load time. An added latency of just one second per page request may also seem affordable; however, also consider the rate of incoming page requests and whether this one-second delay will cause PHP processes to pile up in heavy traffic, thus driving up your server load. Adding more and bigger virtual servers to your stack does not make this I/O latency factor disappear either. The same can be said for disk I/O: virtual disks will always be slower than physical local physical disks, no matter how much CPU and memory the virtual server has been allocated.

However, virtual servers have the advantage of being “elastic,” which means it’s easier to quickly scale horizontally (by adding more servers). Also when dedicated hardware breaks, you have to stop and fix it, unless you have a lot of hot spare servers in the rack, and as we all know, actual “hot spare” hardware is really just a fantasy that sys admins dream about and never actually get.

Avoiding Calling External Web Services

A web server killer we see quite often is custom Drupal modules that call out to an external web service and that external service is slow or unresponsive. This kind of issue can quickly render your web server totally unresponsive to page requests because soon all PHP processes are tied up waiting on an external service that isn’t answering. The root cause is that PHP’s default_socket_timeout defaults to a generous 60 seconds, so each of your PHP processes will block a full minute waiting for a packet that isn’t coming.

511