Advanced PHP Programming

12

Interacting with Databases

RELATIONAL DATABASE MANAGEMENT SYSTEMS (RDBMSS) ARE CRITICAL to modern

applications:They provide powerful and generalized tools for storing and managing persistent data and allow developers to focus more on the core functionality of the applications they develop.

Although RDBMSs reduce the effort required, they still do require some work. Code needs to be written to interface the application to the RDBMS, tables managed by the RDBMS need to be properly designed for the data they are required to store, and queries that operate on these tables need to be tuned for best performance.

Hard-core database administration is a specialty in and of itself, but the pervasiveness of RDBMSs means that every application developer should be familiar enough with how database systems work to spot the good designs and avoid the bad ones.

Database Terminology

The term database is commonly used to refer to both various collections of persistent data and systems that manage persistent collections of data. This usage is often fine for general discussions on databases; however, it can be lacking in a more detailed discussion.

Here are a few technical definitions to help sort things out:

database A collection of persistent data.

database management system (DBMS) A system for managing a database that takes care of things such as controlling access to the data, managing the disk-level representation of the data, and so on.

relational database A database that is organized in tables.

relational database management system (RDBMS) A DBMS that manages relational databases. The results of queries made on databases in the system are returned as tables.

table A collection of data that is organized into two distinct parts: a single header that defines the name and type of columns of data and zero or more rows of data.

For a complete glossary of database terms, see http://www.ocelot.ca/glossary.htm.

300 Chapter 12 Interacting with Databases

Database optimization is important because interactions with databases are commonly the largest bottleneck in an application.

Before you learn about how to structure and tune queries, it’s a good idea to learn about database systems as a whole.This chapter reviews how database systems work, from the perspective of understanding how to design efficient queries.This chapter also provides a quick survey of data access patterns, covering some common patterns for mapping PHP data structures to database data. Finally, this chapter looks at some tuning techniques for speeding database interaction.

Understanding How Databases and Queries

Work

An RDBMS is a system for organizing data into tables.The tables are comprised of rows, and the rows have a specific format. SQL (originally Structured Query Language; now a name without any specific meaning) provides syntax for searching the database to extract data that meets particular criteria. RDBMSs are relational because you can define relationships between fields in different tables, allowing data to be broken up into logically separate tables and reassembled as needed, using relational operators.

The tables managed by the system are stored in disk-based data files. Depending on the RDBMS you use, there may be a one-to-one, many-to-one, or one-to-many relationship between tables and their underlying files.

The rows stored in the tables are in no particular order, so without any additional infrastructure, searching for an item in a table would involve looking through every row in the table to see whether it matches the query criteria.This is known as a full table scan and, as you can imagine, is very slow as tables grow in size.

To make queries more efficient, RDBMSs implement indexes. An index is, as the name implies, a structure to help look up data in a table by a particular field. An index is basically a special table, organized by key, that points to the exact position for rows of that key.The exact data structure used for indexes vary from RDBMS to RDBMS. (Indeed, many allow you to choose the particular type of index from a set of supported algorithms.)

Figure 12.1 shows a sample database lookup on a B-tree–style index. Note that after doing an efficient search for the key in the index, you can jump to the exact position of the matching row.

A database table usually has a primary key. For our purposes, a primary key is an index on a set of one or more columns.The columns in the index must have the following properties:The columns cannot contain null, and the combination of values in the columns must be unique for each row in the table. Primary keys are a natural unique index, meaning that any key in the index will match only a single row.

302 Chapter 12 Interacting with Databases

You can access data from multiple tables by joining them on a common field.When you join tables, it is especially critical to use indexes. For example, say you have a table called users:

CREATE TABLE users ( userid int(11) NOT NULL,

username varchar(30) default NULL, password varchar(10) default NULL, firstname varchar(30) default NULL, lastname varchar(30) default NULL, salutation varchar(30) default NULL, countrycode char(2) NOT NULL default ‘us’

);

and a table called countries:

CREATE TABLE countries ( countrycode char(2) default NULL, name varchar(60) default NULL, capital varchar(60) default NULL

);

Now consider the following query, which selects the username and country name for an individual user by user ID:

SELECT username, name

FROM users, countries

WHERE userid = 1

AND users.countrycode = countries.countrycode;

If you have no indexes, you must do a full table scan of the products of both tables to complete the query.This means that if users has 100,000 rows and countries contains 239 rows, 23,900,000 joined rows must be examined to return the result set. Clearly this is a bad procedure.

To make this lookup more efficient, you need to add indexes to the tables. A first start is to add primary keys to both tables. For users, userid is a natural choice, and for countries the two-letter International Organization for Standardization (ISO) code will do. Assuming that the field that you want to make the primary key is unique, you can use the following after table creation:

mysql> alter table users add primary key(userid);

Or, during creation, you can use the following:

CREATE TABLE countries (

countrycode char(2) NOT NULL default ‘us’, name varchar(60) default NULL,

capital varchar(60) default NULL, PRIMARY KEY (countrycode)

);

Now when you do a lookup, you first perform a lookup by index on the users table to find the row with the matching user ID.Then you take that user’s countrycode and perform a lookup by key with that in countries.The total number of rows that need to be inspected is 1.This is a considerable improvement over inspecting 23.9 million rows.

Query Introspection with EXPLAIN

Determining the query path in the previous example was done simply with logical deduction.The problem with using logic to determine the cost of queries is that you and the database are not equally smart. Sometimes the query optimizer in the database makes bad choices. Sometimes people make bad choices. Because the database will be performing the query, its opinion on how the query will be run is the one that counts the most. Manual inspection is also time-consuming and difficult, especially as queries become complex.

Fortunately, most RDBMSs provide the EXPLAIN SQL syntax for query execution path introspection. EXPLAIN asks the query optimizer to generate an execution plan for the query.The exact results of this vary from RDBMS to RDBMS, but in general EXPLAIN returns the order in which the tables will be joined, any indexes that will used, and an approximate cost of each part of the query (the number of rows in the tables being queried and so on).

Let’s look at a real-world example. On a site I used to work on, there was a visit table that tracked the number of visits a user had made and the last time they visited.The table looked like this:

CREATE TABLE visits ( userid int not null, last_visit timestamp,

count int not null default 0, primark key(userid)

);

The normal access pattern for this table was to find the visit count and last visit for the user on login (so that a welcome message such as “You last logged in on…” can be displayed). Using EXPLAIN to inspect that query shows the following:

This shows the table being accessed (visit), the type of join being performed (const, because it is a single-table query and no join is happening), the list of possible keys that could be used (only PRIMARY on the table is eligible), the key that it has picked from that

304 Chapter 12 Interacting with Databases

list, the length of the key, and the number of rows it thinks it will examine to get the result.This is an efficient query because it is keyed off the primary key visits.

As this application evolves, say that I decide that I would like to use this information to find the number of people who have logged in in the past 24 hours. I’d do this with the following query:

SELECT count(*) FROM visits WHERE last_visit > NOW() - 86400;

EXPLAIN for this query generates the following:

mysql> explain select count(*) from visits where last_visit > now() - 86400;

Notice here that the query has no keys that it can use to complete the query, so it must do a complete scan of the table, examining all 511,517 rows and comparing them against the WHERE clause. I could improve this performance somewhat by adding an index on visits. When I do this, I get the following results:

The new index is successfully used, but it is of limited effectiveness (because, apparently, a large number of users log in every day). A more efficient solution for this particular problem might be to add a counter table per day and have this updated for the day on a user’s first visit for the day (which can be confirmed from the user’s specific entry in visits):

CREATE TABLE visit_summary ( day date,

count int, primary key(date)

) ;

Finding Queries to Profile

One of the hardest parts of tuning a large application is finding the particular code sections that need to be tuned.Tuning databases is no different:With hundreds or thousands of queries in a system, it is critical that you be able to focus your effort on the most critical bottlenecks.

Every RDBMS has its own techniques for finding problem queries. In MySQL the easiest way to spot trouble queries is with the slow query log.The slow query log is enabled with a triumvirate of settings in the MySQL configuration file. Basic slow-query logging is enabled with this:

log-slow-queries = /var/lib/mysql/slow-log

If no location is specified, the slow query log will be written to the root of the data directory as server-name-slow.log.To set a threshold for how long a query must take (in seconds) to be considered slow, you use this setting:

or

long_query_time=5 (MySQL 4+)

Finally, if you would also like MySQL to automatically log any query that does not use an index, you can set this:

log-long-format (MySQL 3,4.0)

or

log-queries-not-using-indexes (MySQL 4.1+)

Then, whenever a query takes longer than the long_query_time setting or fails to use an index, you get a log entry like this:

select UNIX_TIMESTAMP(NOW())-UNIX_TIMESTAMP(MAX(last_visit)) FROM visits;

#User@Host: user[user] @ db.omniti.net [10.0.0.1]

#Query_time: 6 Lock_time: 0 Rows_sent: 1 Rows_examined: 511517

This tells you what query was run, how much time it took to complete, in seconds, how many rows it returned, and how many rows it had to inspect to complete its task.

The slow query log is the first place I start when tuning a new client’s site. I usually start by setting long_query_time to 10 seconds, fix or replace every query that shows up, and then drop the amount of time and repeat the cycle.The goal for any production Web site should be to be able to set long_query_time to 1 second and have the log be completely free of queries (this assumes that you have no data-mining queries running against your production database; ignore those if you do).

The mysqldumpslow tool is very handy for reading the slow query log. It allows you to summarize and sort the results in the slow query log for easier analysis.

306 Chapter 12 Interacting with Databases

Queries are grouped into entries that display the number of times the query was placed in the slow query log, the total time spent executing the group of queries, and so on.

Here’s an example:

Count: 4 Time=0.25s (1s) Lock=0.00s (0s) Rows=3.5 (14), root[root]@localhost

SELECT * FROM users LIMIT N

Count: 5 Time=0.20s (1s) Lock=0.00s (0s) Rows=5.0 (25), root[root]@localhost

SELECT * FROM users

The tool accepts options to control how the queries are sorted and reported.You can run mysqldumpslow --help for more information on the options.

Logging of non-indexed queries can also be enlightening, but I tend not to leave it on. For queries running on very small tables (a couple hundred rows), it is often just as fast—if not faster—for the RDBMS to avoid the index as to use it.Turning on

log-long-format is a good idea when you come into a new environment (or when you need to do a periodic audit of all the SQL running in an application), but you do not want these queries polluting your logs all the time.

Database Access Patterns

Database access patterns define the way you will interact with an RDBMS in PHP code. At a simplistic level, this involves determining how and where SQL will appear in the code base.The span of philosophies on this is pretty wide. On one hand is a camp of people who believe that data access is such a fundamental part of an application that SQL should be freely mixed with PHP code whenever a query needs to be performed. On the opposite side are those who feel that SQL should be hidden from developers and that all database access should be contained within deep abstraction layers.

I tend not to agree with either of these points of view.The problem with the first is largely a matter of refactoring and reuse. Just as with PHP functions, if you have similar code repeated throughout a code base, for any structural changes that need to be made, you will need to track down every piece of code that might be affected.This creates unmanageable code.

The problem I have with the abstraction viewpoint is that abstractions all tend to be lossy.That is, when you wrap something in an abstraction layer, you inevitably lose some of the fine-grained control that you had in the native interface. SQL is a powerful language and is common enough that developers should understand and use it comfortably.

Being a centrist on this issue still leaves a good bit of room for variation.The following sections present four database access patterns—ad hoc queries, the Active Record pattern, the Mapper pattern, and the Integrated Mapper Pattern—that apply to both simplistic database query needs and to more complex object-data mapping requirements.

Ad Hoc Queries

Ad hoc queries are by definition not a pattern, but they can still be useful in many contexts. An ad hoc query is a query that is written to solve a particular problem in a particular spot of code. For example, the following snippet of procedural code to update the country of a user in the users table has an ad hoc character to it:

function setUserCountryCode($userid, $countrycode)

{

$dbh = new DB_Mysql_Test;

$query = “UPDATE users SET countrycode = :1 WHERE userid = :2”; $dbh->prepare($query)->execute($countrycode, $userid);

}

Ad hoc queries are not inherently bad. In fact, because an ad hoc query is usually designed to handle a very particular task, it has the opportunity to be tuned (at a SQL level) much more highly than code that serves a more general purpose.The thing to be aware of with ad hoc queries is that they can proliferate through a code base rapidly.You start with a special-purpose ad hoc query here and there, and then suddenly you have 20 different queries spread throughout your code that modify the countrycode column of users.That is a problem because it is very difficult to track down all such queries if you ever need to refactor users.

That having been said, I use ad hoc queries quite frequently, as do many professional coders.The trick to keeping them manageable is to keep them in centralized libraries, according to the task they perform and the data they alter. If all your queries that modify users are contained in a single file, in a central place, refactoring and management is made much easier.

The Active Record Pattern

Often you might have classes that directly correspond to rows in a database.With such a setup, it is nice to directly tie together access to the object with the underlying database access.The Active Record pattern encapsulates all the database access for an object into the class itself.

The distinguishing factor of the Active Record pattern is that the encapsulating class will have an insert(), an update(), and a delete() method for synchronizing an object with its associated database row. It should also have a set of finder methods to create an object from its database row, given an identifier.

Here is an example of an implementation of the User class that corresponds with the user database table we looked at earlier:

require_once “DB.inc”;

class User { public $userid; public $username;