Robert I. Kabacoff - R in action

You have to give R more information to find it.

Now that you know how to generate descriptive statistics for the data as a whole, let’s review how to obtain statistics for subgroups of the data.

7.1.2Descriptive statistics by group

When comparing groups of individuals or observations, the focus is usually on the descriptive statistics of each group, rather than the total sample. Again, there are several ways to accomplish this in R. We’ll start by getting descriptive statistics for each level of transmission type.

In chapter 5, we discussed methods of aggregating data. You can use the aggregate() function (section 5.6.2) to obtain descriptive statistics by group, as shown in the following listing.

Listing 7.6 Descriptive statistics by group using aggregate()

> aggregate(mtcars[vars], by=list(am=mtcars$am), mean) am mpg hp wt

10 17.1 160 3.77

21 24.4 127 2.41

> aggregate(mtcars[vars], by=list(am=mtcars$am), sd) am mpg hp wt

10 3.83 53.9 0.777

21 6.17 84.1 0.617

Note the use of list(am=mtcars$am). If you had used list(mtcars$am), the am column would have been labeled Group.1 rather than am. You use the assignment to provide a more useful column label. If you have more than one grouping variable, you can use code like by=list(name1=groupvar1, name2=groupvar2, … , groupvarN).

Unfortunately, aggregate() only allows you to use single value functions such as mean, standard deviation, and the like in each call. It won’t return several statistics at once. For that task, you can use the by() function. The format is

by(data, INDICES, FUN)

where data is a data frame or matrix, INDICES is a factor or list of factors that define the groups, and FUN is an arbitrary function. This next listing provides an example.

Listing 7.7 Descriptive statistics by group using by()

>dstats <- function(x)(c(mean=mean(x), sd=sd(x)))

>by(mtcars[vars], mtcars$am, dstats)

EXTENSIONS

The doBy package and the psych package also provide functions for descriptive statistics by group. Again, they aren’t distributed in the base installation and must be installed before first use. The summaryBy() function in the doBy package has the format

summaryBy(formula, data=dataframe, FUN=function)

where the formula takes the form

var1 + var2 + var3 + ... + varN ~ groupvar1 + groupvar2 + … + groupvarN

Variables on the left of the ~ are the numeric variables to be analyzed and variables on the right are categorical grouping variables. The function can be any built-in or usercreated R function. An example using the mystats() function you created in section 7.2.1 is shown in the following listing.

Listing 7.8 Summary statistics by group using summaryBy() in the doBy package

>library(doBy)

>summaryBy(mpg+hp+wt~am, data=mtcars, FUN=mystats)

am mpg.n mpg.mean mpg.stdev mpg.skew mpg.kurtosis hp.n hp.mean hp.stdev

The describe.by() function contained in the psych package provides the same descriptive statistics as describe, stratified by one or more grouping variables, as you can see in the following listing.

Listing 7.9 Summary statistics by group using describe.by() in the psych package

>library(psych)

>describe.by(mtcars[vars], mtcars$am) group: 0

Unlike the previous example, the describe.by() function doesn’t allow you to specify an arbitrary function, so it’s less generally applicable. If there’s more than one grouping variable, you can write them as list(groupvar1, groupvar2, … , groupvarN). But this will only work if there are no empty cells when the grouping variables are crossed.

Finally, you can use the reshape package described in section 5.6.3 to derive descriptive statistics by group in a flexible way. (If you haven’t read that section, I suggest you review it before continuing.) First, you melt the data frame using

dfm <- melt(dataframe, measure.vars=y, id.vars=g)

where dataframe contains the data, y is a vector indicating the numeric variables to be summarized (the default is to use all), and g is a vector of one or more grouping variables. You then cast the data using

cast(dfm, groupvar1 + groupvar2 + … + variable ~ ., FUN)

where the grouping variables are separated by + signs, the word variable is entered exactly as is, and FUN is an arbitrary function.

In the final example of this section, we’ll apply the reshape approach to obtaining descriptive statistics for each subgroup formed by transmission type and number of cylinders. For descriptive statistics, we’ll get the sample size, mean, and standard deviation. The code and results are shown in the following listing.

Listing 7.10 Summary statistics by group via the reshape package

>library(reshape)

>dstats <- function(x)(c(n=length(x), mean=mean(x), sd=sd(x)))

>dfm <- melt(mtcars, measure.vars=c("mpg", "hp", "wt"),

id.vars=c("am", "cyl"))

> cast(dfm, am + cyl + variable ~ ., dstats)

Personally, I find this approach the most compact and appealing. Data analysts have their own preferences for which descriptive statistics to display and how they like to see them formatted. This is probably why there are many variations available. Choose the one that works best for you, or create your own!

7.1.3Visualizing results

Numerical summaries of a distribution’s characteristics are important, but they’re no substitute for a visual representation. For quantitative variables you have histograms (section 6.3), density plots (section 6.4), box plots (section 6.5), and dot plots (section 6.6). They can provide insights that are easily missed by reliance on a small set of descriptive statistics.

The functions considered so far provide summaries of quantitative variables. The functions in the next section allow you to examine the distributions of categorical variables.

7.2Frequency and contingency tables

In this section, we’ll look at frequency and contingency tables from categorical variables, along with tests of independence, measures of association, and methods for graphically displaying results. We’ll be using functions in the basic installation, along with functions from the vcd and gmodels package. In the following examples, assume that A, B, and C represent categorical variables.

The data for this section come from the Arthritis dataset included with the vcd package. The data are from Kock & Edward (1988) and represent a double-blind clinical trial of new treatments for rheumatoid arthritis. Here are the first few observations:

>library(vcd)

>head(Arthritis)

Treatment (Placebo, Treated), Sex (Male, Female), and Improved (None, Some, Marked) are all categorical factors. In the next section, we’ll create frequency and contingency tables (cross-classifications) from the data.

7.2.1Generating frequency tables

R provides several methods for creating frequency and contingency tables. The most important functions are listed in table 7.1.

Table 7.1 Functions for creating and manipulating contingency tables

In the following sections, we’ll use each of these functions to explore categorical variables. We’ll begin with simple frequencies, followed by two-way contingency tables, and end with multiway contingency tables. The first step is to create a table using either the table() or the xtabs() function, then manipulate it using the other functions.

ONE-WAY TABLES

You can generate simple frequency counts using the table() function. Here’s an example:

>mytable <- with(Arthritis, table(Improved))

>mytable

Improved

None Some Marked

42 14 28

You can turn these frequencies into proportions with prop.table():

> prop.table(mytable) Improved

None Some Marked 0.500 0.167 0.333

or into percentages, using prop.table()*100:

> prop.table(mytable)*100 Improved

None Some Marked 50.0 16.7 33.3

Here you can see that 50 percent of study participants had some or marked improvement (16.7 + 33.3).

TWO-WAY TABLES

For two-way tables, the format for the table() function is

mytable <- table(A, B)

where A is the row variable, and B is the column variable. Alternatively, the xtabs() function allows you to create a contingency table using formula style input. The format is

mytable <- xtabs(~ A + B, data=mydata)

where mydata is a matrix or data frame. In general, the variables to be cross-classified appear on the right of the formula (that is, to the right of the ~) separated by + signs. If a variable is included on the left side of the formula, it’s assumed to be a vector of frequencies (useful if the data have already been tabulated).

For the Arthritis data, you have

>mytable <- xtabs(~ Treatment+Improved, data=Arthritis)

>mytable

Improved

Treatment None Some Marked

Placebo 29 7 7

Treated 13 7 21

You can generate marginal frequencies and proportions using the margin.table() and prop.table() functions, respectively. For row sums and row proportions, you have

> margin.table(mytable, 1) Treatment

Placebo Treated

4341

>prop.table(mytable, 1) Improved

Treatment None Some Marked

Placebo 0.674 0.163 0.163

Treated 0.317 0.171 0.512

The index (1) refers to the first variable in the table() statement. Looking at the table, you can see that 51 percent of treated individuals had marked improvement, compared to 16 percent of those receiving a placebo.

For column sums and column proportions, you have

> margin.table(mytable, 2) Improved

None Some Marked 42 14 28

> prop.table(mytable, 2) Improved

Treatment None Some Marked

Placebo 0.690 0.500 0.250

Treated 0.310 0.500 0.750

Here, the index (2) refers to the second variable in the table() statement. Cell proportions are obtained with this statement:

> prop.table(mytable) Improved

Treatment None Some Marked Placebo 0.3452 0.0833 0.0833 Treated 0.1548 0.0833 0.2500

You can use the addmargins() function to add marginal sums to these tables. For example, the following code adds a sum row and column:

> addmargins(mytable) Improved

Treatment None Some Marked Sum

> addmargins(prop.table(mytable))

When using addmargins(), the default is to create sum margins for all variables in a table. In contrast:

> addmargins(prop.table(mytable, 1), 2) Improved

Treatment None Some Marked Sum Placebo 0.674 0.163 0.163 1.000 Treated 0.317 0.171 0.512 1.000

adds a sum column alone. Similarly,

> addmargins(prop.table(mytable, 2), 1)

adds a sum row. In the table, you see that 25 percent of those patients with marked improvement received a placebo.

NOTE The table() function ignores missing values (NAs) by default. To include NA as a valid category in the frequency counts, include the table option useNA="ifany".

A third method for creating two-way tables is the CrossTable() function in the gmodels package. The CrossTable() function produces two-way tables modeled after PROC FREQ in SAS or CROSSTABS in SPSS. See listing 7.11 for an example.

Listing 7.11 Two-way table using CrossTable

>library(gmodels)

>CrossTable(Arthritis$Treatment, Arthritis$Improved)

The CrossTable() function has options to report percentages (row, column, cell); specify decimal places; produce chi-square, Fisher, and McNemar tests of independence; report expected and residual values (Pearson, standardized, adjusted standardized); include missing values as valid; annotate with row and column titles; and format as SAS or SPSS style output. See help(CrossTable) for details.

If you have more than two categorical variables, you’re dealing with multidimensional tables. We’ll consider these next.

MULTIDIMENSIONAL TABLES

Both table() and xtabs() can be used to generate multidimensional tables based on three or more categorical variables. The margin.table(), prop.table(), and addmargins() functions extend naturally to more than two dimensions. Additionally, the ftable() function can be used to print multidimensional tables in a compact and attractive manner. An example is given in listing 7.12.

The code in . produces cell frequencies for the three-way classification. The code also demonstrates how the ftable() function can be used to print a more compact and attractive version of the table.

The code in 3produces the marginal frequencies for Treatment, Sex, and Improved. Because you created the table with the formula ~Treatement+Sex+Improve, Treatment is referred to by index 1, Sex is referred to by index 2, and Improve is referred to by index 3.

The code in $ produces the marginal frequencies for the Treatment x Improved classification, summed over Sex. The proportion of patients with None, Some, and Marked improvement for each Treatment x Sex combination is provided in /. Here you see that 36 percent of treated males had marked improvement, compared to 59 percent of treated females. In general, the proportions will add to one over the indices not included in the prop.table() call (the third index, or Improve in this case). You can see this in the last example, where you add a sum margin over the third index.

If you want percentages instead of proportions, you could multiply the resulting table by 100. For example:

ftable(addmargins(prop.table(mytable, c(1, 2)), 3)) * 100

would produce this table:

While contingency tables tell you the frequency or proportions of cases for each combination of the variables that comprise the table, you’re probably also interested in whether the variables in the table are related or independent. Tests of independence are covered in the next section.