In my data, i have two raters that rated multiple scores to do a test-retest analysis. The method fulfill the requirements for test-retest (short period between both visits)

I would like to calculate an ICC for each score.
But my point is that, i only see test-retest analysis for one rater, but can’t take into account when there is more than one rater (i have two raters).

Here is a code that can reproduce my data for one score:

library(irr)
library(psych)
library(tidyr)


# Data
data <- data.frame(
  subject = rep(1:10, each = 4),  # 10 subjects rated at two timepoints by 2 raters
  rater = rep(rep(1:2, each = 2), times = 10),
  Moment = rep(1:2, times = 20),
  Score = c(23, 24, 23, 25, 22, 22, 21, 23, 20, 21, 24, 25, 26, 25, 24, 24, 22, 22, 23, 24,
            25, 26, 26, 27, 24, 24, 25, 26, 23, 23, 24, 25, 22, 23, 21, 22, 23, 24, 22, 22)
)

# Wider
data_wide <- data %>%
  pivot_wider(names_from = Moment, values_from = Score, names_prefix = "Moment_")

# ICC by rater
icc_results <- data_wide %>%
  group_by(rater) %>%
  summarise(ICC = icc(data.frame(Moment_1, Moment_2), model = "twoway", type = "agreement")$value)

print(icc_results)
# A tibble: 2 × 2
  rater   ICC
  <int> <dbl>
1     1 0.886
2     2 0.778

I can have a result for each rater, but i would like to know if it’s possible to have a “global” ICC that takes into account that there are 2 raters. Because if i do not group_by there, i would have an ICC that takes each observation for a different individual, without having the information of subject/rater