Intraclass Correlation Coefficient

Definition

Intraclass correlation coefficient (ICC) measures how consistent repeated measurements are within the same subject.

It asks:

Of the total variation in a measured variable, how much comes from real between-person differences rather than within-person measurement fluctuation?

For repeated exposure measurements, ICC is a reliability measure. If ICC is high, one measurement can better represent a person’s usual level. If ICC is low, a single measurement is noisy and exposure-disease associations may be attenuated by regression dilution bias.

Data Shape

For a simple ICC calculation in R, use a wide table:

df <- data.frame(
  id = 1:5,
  overall_t1 = c(35, 60, 20, 48, 70),
  overall_t2 = c(37, 58, 22, 47, 68),
  overall_t3 = c(34, 62, 21, 50, 72),
  overall_t4 = c(36, 59, 19, 49, 69)
)

Rows are people. Columns are repeated measurements of the same variable.

For example, overall_t1 to overall_t4 could be four repeated measurements of accelerometer-derived overall activity.

R Function

Use irr::icc() when you want explicit control over the ICC model:

library(irr)

activity_mat <- df[, c("overall_t1", "overall_t2", "overall_t3", "overall_t4")]

irr::icc(
  activity_mat,
  model = "twoway",
  type  = "agreement",
  unit  = "single"
)

Input:

• a numeric matrix or data frame;
• each row is one subject;
• each column is one repeated measurement.

Output:

• ICC estimate;
• F statistic;
• p-value;
• 95% confidence interval.

How To Read ICC1, ICC2, ICC3

Some functions, especially psych::ICC(), report several ICC types:

ICC1
ICC2
ICC3
ICC1k
ICC2k
ICC3k

They are not interchangeable.

ICC1 is a one-way random-effects ICC. It does not explicitly model fixed measurement occasions or raters.

ICC2 is a two-way random-effects ICC. It treats subjects and measurement occasions or raters as random, supporting generalization to other similar raters or occasions.

ICC3 is a two-way mixed-effects ICC. It treats subjects as random but the measurement occasions or raters as fixed. This is appropriate when the specific measurement setup is the one being evaluated.

What The k Means

The versions without k evaluate a single measurement:

ICC1, ICC2, ICC3

The versions with k evaluate the average of k measurements:

ICC1k, ICC2k, ICC3k

This distinction matters.

If a cohort usually has only one exposure measurement per participant, the relevant quantity is the single-measure ICC. If the analysis averages four repeated measurements per person, the relevant quantity is an average-measure ICC.

Average-measure ICC is usually higher because averaging repeated measurements reduces random within-person variation.

Which One To Use

Match the ICC to the scientific question.

If the question is:

How reliable is one 7-day accelerometer measurement as a proxy for usual activity?

then use:

irr::icc(activity_mat, model = "twoway", type = "agreement", unit = "single")

If the question is:

How reliable is the average of four repeated accelerometer measurements?

then use:

irr::icc(activity_mat, model = "twoway", type = "agreement", unit = "average")

For the UK Biobank accelerometer reproducibility paper, the important interpretation is the single-measure reliability, because most UKB participants have only one 7-day accelerometer assessment.

Formula Intuition

A simple random-intercept framing is:

measurement_ij = overall mean + subject_i + error_ij

The ICC is approximately:

between-person variance / (between-person variance + within-person variance)

So ICC is high when people differ from each other in stable ways and repeated measurements within the same person are close.

ICC is low when repeated measurements within the same person fluctuate strongly.

In Papers

Zisou et al. 2026 used ICCs to assess the long-term reproducibility of UK Biobank accelerometer-derived physical activity and sleep phenotypes. Overall activity had good reproducibility (ICC about 0.75), while other phenotypes were moderate (about 0.58-0.69).

The paper then used the ICC for daily step count to correct the step count-CHD association for regression dilution bias. The observed association was weaker than the association estimated for usual step count, showing why measurement reliability changes epidemiological interpretation.