2.6  One-Way Analysis of Variance

We now consider models where the predictors are categorical variables or factors with a discrete number of levels. To illustrate the use of these models we will group the index of social setting (and later the index of family planning effort) into discrete categories.

2.6.1  The One-Way Layout

Table 2.10 shows the percent decline in the CBR for the 20 countries in our illustrative dataset, classified according to the index of social setting in three categories: low (under 70 points), medium (70-79) and high (80 or more).

It will be convenient to modify our notation to reflect the one-way layout of the data explicitly. Let k denote the number of groups or levels of the factor, n_i denote the number of observations in group i, and let y_ij denote the response for the j-th unit in the i-th group, for j = 1,,n_i, and i = 1,,k. In our example k = 3 and y_ij is the CBR decline in the j-th country in the i-th category of social setting, with i = 1,2,3; j = 1, , n_i; n₁ = 7, n₂ = 5 and n₃ = 8).

2.6.2  The One-Factor Model

As usual, we treat y_ij as a realization of a random variable Y_ij ~ N(m_ij, s²), where the variance is the same for all observations. In terms of the systematic structure of the model, we assume that

Before we proceed further, it is important to note that the model as written is not identified. We have essentially k groups but have introduced k+1 linear parameters. The solution is to introduce a constraint, and there are several ways in which we could proceed.

One approach is to set m = 0 (or simply drop m). If we do this, the a_i's become cell means, with a_i representing the expected response in group i. While simple and attractive, this approach does not generalize well to models with more than one factor.

Our preferred alternative is to set one of the a_i's to zero. Conventionally we set a₁ = 0, but any of the groups could be chosen as the reference cell or level. In this approach m becomes the expected response in the reference cell, and a_i becomes the effect of level i of the factor, compared to the reference level.

A third alternative is to require the group effects to add-up to zero, so a_i = 0. In this case m represents some sort of overall expected response, and a_i measures the extent to which responses at level i of the factor deviate from the overall mean. Some statistics texts refer to this constraint as the `usual' restrictions, but I think the reference cell method is now used more widely in social research.

A variant of the `usual' restrictions is to require a weighted sum of the effects to add up to zero, so w_i a_i = 0. The weights are often taken to be the number of observations in each group, so w_i = n_i. In this case m is a weighted average representing the expected response, and a_i is, as before, the extent to which responses at level i of the factor deviate from the overall mean.

Each of these parameterizations can easily be translated into one of the others, so the choice can rest on practical considerations. The reference cell method is easy to implement in a regression context and the resulting parameters have a clear interpretation.

2.6.3  Estimates and Standard Errors

The model in Equation 2.18 is a special case of the generalized linear model, where the design matrix X has k+1 columns: a column of ones representing the constant, and k columns of indicator variables, say x₁, , x_k, where x_i takes the value one for observations at level i of the factor and the value zero otherwise.

Note that the model matrix as defined so far is rank deficient, because the first column is the sum of the last k. Hence the need for constraints. The cell means approach is equivalent to dropping the constant, and the reference cell method is equivalent to dropping one of the indicator or dummy variables representing the levels of the factor. Both approaches are easily implemented. The other two approaches, which set to zero either the unweighted or weighted sum of the effects, are best implemented using Lagrange multipliers and will not be considered here.

Parameter estimates, standard errors and t ratios can then be obtained from the general results of Sections 2.2 and 2.3. You may be interested to know that the estimates of the regression coefficients in the one-way layout are simple functions of the cell means. Using the reference cell method,

Table 2.11 shows the estimates for our sample data. We expect a CBR decline of almost 8% in countries with low social setting (the reference cell). Increasing social setting to medium or high is associated with additional declines of one and 16 percentage points, respectively, compared to low setting.

Looking at the t ratios we see that the difference between medium and low setting is not significant, so we accept H₀: a₂ = 0, whereas the difference between high and low setting, with a t-ratio of 3.38 on 17 d.f. and a two-sided P-value of 0.004, is highly significant, so we reject H₀:a₃ = 0. These t-ratios test the significance of two particular contrasts: medium vs. low and high vs. low. In the next subsection we consider an overall test of the significance of social setting.

2.6.4  The One-Way Anova Table

Fitting the model with social setting treated as a factor reduces the RSS from 2650.2 (for the null model) to 1456.4, a gain of 1193.8 at the expense of two degrees of freedom (the two a's). We can contrast this gain with the remaining RSS of 1456.4 on 17 d.f. The calculations are laid out in Table 2.12, and lead to an F-test of 6.97 on 2 and 17 d.f., which has a P-value of 0.006. We therefore reject the hypothesis H₀: a₂ = a₃ = 0 of no setting effects, and conclude that the expected response depends on social setting.

Having established that social setting has an effect on CBR decline, we can inspect the parameter estimates and t-ratios to learn more about the nature of the effect. As noted earlier, the difference between high and low settings is significant, while that between medium and low is not.

It is instructive to calculate the Wald test for this example. Let a = (a₂,a₃) denote the two setting effects. The estimate and its variance-covariance matrix, calculated using the general results of Section 2.2, are

2.6.5  The Correlation Ratio

Note from Table 2.12 that the model treating social setting as a factor with three levels has reduced the RSS by 1456.6 out of 2650.2, thereby explaining 45.1%. The square root of the proportion of variance explained by a discrete factor is called the correlation ratio, and is often denoted h. In our example [^(h)] = 0.672.

If the factor has only two categories the resulting coefficient is called the point-biserial correlation, a measure often used in psychometrics to correlate a test score (a continuous variable) with the answer to a dichotomous item (correct or incorrect). Note that both measures are identical in construction to Pearson's correlation coefficient. The difference in terminology reflects whether the predictor is a continuous variable with a linear effect or a discrete variable with two or more than two categories.

Continue with 2.7. Two-Way Analysis of Variance
Copyright © Germán Rodríguez, 1993-2000. Please send feedback to grodri@princeton.edu
Conversion from LaTeX was done using T_TH, version 2.34.