WWS509 - Lecture Notes

B.5 Poisson Errors and Link Log

Let us now apply the general theory to the Poisson case, with emphasis on the log link function.

B.5.1 The Poisson Distribution

A Poisson random variable has probability distribution function

f_i(y_i) =

e^-m_i m_i^y_i

y_i!

(B.20)

for y_i = 0, 1, 2, . The moments are

E(Y_i) = var(Y_i) = m_i.

Let us verify that this distribution belongs to the exponential family as defined by Nelder and Wedderburn (1972). Taking logs we find

logf_i(y_i) = y_i log(m_i) - m_i - log(y_i!).

Looking at the coefficient of y_i we see immediately that the canonical parameter is

q_i = log(m_i),

(B.21)

and therefore that the canonical link is the log. Solving for m_i we obtain the inverse link

m_i = e^q_i,

and we see that we can write the second term in the p.d.f. as

b(q_i) = e^q_i.

The last remaining term is a function of y_i only, so we identify

c(y_i,f) = -log(y_i!).

Finally, note that we can take a_i(f) = f and f = 1, just as we did in the binomial case.

Let us verify the mean and variance. Differentiating the cumulant function b(q_i) we have

m_i = b(q_i) = e^q_i = m_i,

and differentiating again we have

v_i = a_i(f) b(q_i) = e^q_i = m_i.

Note that the mean equals the variance.

B.5.2 Fisher Scoring in Log-linear Models

We now consider the Fisher scoring algorithm for Poisson regression models with canonical link, where we model

h_i = log(m_i).

(B.22)

The derivative of the link is easily seen to be

dh_i

dm_i

m_i

Thus, the working dependent variable has the form

z_i = h_i +

y_i - m_i

m_i

(B.23)

The iterative weight is

w_i

1 /

b(q_i) (

dh_i

dm_i

)²

1 /

m_i

m_i²

and simplifies to

w_i = m_i.

(B.24)

Note again that the weight is inversely proportional to the variance of the working dependent variable.

B.5.3 The Poisson Deviance

Let [^(m_i)] denote the m.l.e. of m_i under the model of interest and let [(m_i)\tilde] = y_i denote the m.l.e. under the saturated model. From first principles, the deviance is

[ y_i log(y_i) - y_i - log(y_i!)

- y_i log(

^
m_i

) +

^
m_i

+ log(y_i!)].

Note that the terms on y_i! cancel out. Collecting terms on y_i we have

D = 2

[ y_i log(

y_i

^
m_i

) - (y_i -

^
m_i

)].

(B.25)

The similarity of the Poisson and Binomial deviances should not go unnoticed. Note that the first term in the Poisson deviance has the form

D = 2

o_i log(

o_i

e_i

which is identical to the Binomial deviance. The second term is usually zero. To see this point, note that for a canonical link the score is

logL

= X(y-m),

and setting this to zero leads to the estimating equations

Xy = X

^
m

In words, maximum likelihood estimation for Poisson log-linear models-and more generally for any generalized linear model with canonical link-requires equating certain functions of the m.l.e.'s (namely X[^(m)]) to the same functions of the data (namely Xy). If the model has a constant, one column of X will consist of ones and therefore one of the estimating equations will be

y_i =

^
m_i

(y_i-

^
m_i

) = 0,

so the last term in the Poisson deviance is zero. This result is the basis of an alternative algorithm for computing the m.l.e.'s known as ``iterative proportional fitting'', see Bishop et al. (1975) for a description.

The Poisson deviance has an asymptotic chi-squared distribution as n with the number of parameters p remaining fixed, and can be used as a goodness of fit test. Differences between Poisson deviances for nested models (i.e. the log of the likelihood ratio test criterion) have asymptotic chi-squared distributions under the usual regularity conditions.