Poisson, offset, and negative binomial

Upon completing this topic

You will be able to distinguish between count, rate, and exposure time, and explain the role of the offset in a Poisson GLM. You will avoid automatically selecting a Poisson model without checking for overdispersion, and you will understand the negative binomial model as an alternative model with different assumptions.

Count and rate are different

The number of events observed over the same period can be expressed as a count. When observation times differ, one must consider the rate rather than comparing counts alone. The offset is a method for reflecting differences in observation opportunity or exposure in the linear predictor of the model.

Questions for the Poisson model

The Poisson structure includes assumptions about the mean–variance relationship of the counts. Even if the count is an integer and non-negative, the Poisson assumption does not automatically hold. One must check for overdispersion, the pattern of zeros, and cluster or time dependence.

Models and result objects

import numpy as np
import statsmodels.api as sm

X = sm.add_constant(data[["exposure_group"]])
poisson = sm.GLM(
    data["count"], X,
    family=sm.families.Poisson(),
    offset=np.log(data["observation_time"]),
).fit()
print(poisson.params)

observation_time must be positive and have defined units; the offset is a contract to include the log of the observation opportunity in the model. The fact that the API has run does not automatically mean that the rate interpretation is correct.

Overdispersion and alternatives

Observed variability may exceed the level allowed by the Poisson assumption. In such cases, the cause should be investigated first. Issues may arise from repeated or clustered observations, omitted covariates, or the data-generating structure, and the negative binomial is not a universal remedy for all causes. When changing models, record both the assumptions and the diagnostic results.

Returning to the research question for interpretation

The coefficients of a count model are on the link scale, and exponentiated values can be read as rate ratios, but this requires knowledge of the units and offset definition. The selection of a specific model should not be taken as confirmation of the biological generative mechanism.

Common failures

• Do not use "count" and "rate" interchangeably.
• Do not omit the offset when observation times differ.
• Do not automatically select a Poisson model simply because the outcome is an integer.
• Do not interpret standard errors without checking for overdispersion.

Key takeaways

• Distinguish between count and rate by including differences in observation opportunity.
• The offset connects exposure to the model.
• Poisson and negative binomial have different assumptions.

Next topic

In the next section, we distinguish between fixed effects and random effects in mixed models.

References

• statsmodels GLM: https://www.statsmodels.org/stable/glm.html
• statsmodels API: https://www.statsmodels.org/stable/api.html
• Bioconductor OSTA experimental design: https://bioconductor.org/books/release/OSTA/pages/bkg-exp-design.html

The count schema and code in this section were written independently by BioStatPy.