I am trying to annotate a ggplot line of best fit for a scatterplot with an arrow.

This replicates an example in Paul Murrell’s book R Graphics (Third Edition), though that example modifies grobs on the grid display list whereas I would like to do it by manipulating the grobs produced by ggplot2. Code for the original example is here (Figure 7.16).

I want to write ‘Line of best fit’ in the plot region and draw an arrow from this, ending on the actual line of best fit.

I have almost got this to work but cannot find the right way to place the end of the arrow as it points to the bottom of the confidence interval ribbon instead of the line itself.

Code so far:

library(ggplot2)
library(grid)

p <- ggplot(mtcars, aes(x = disp, y = mpg)) +
  geom_point() +
  geom_smooth(method = lm, se = TRUE)

gg <- ggplotGrob(p)

# Retrieve the panel grob
panel_grob <- gg[["grobs"]][gg$layout$name == "panel"][[1]]

child_nms <- names(panel_grob[["children"]])

# Which child of the "panel" grob contains "smooth" in its name?
smooth_idx <- which(grepl("smooth", child_nms))
smooth_grob <- panel_grob$children[smooth_idx][[1]]

# Which child of the "smooth" grob contains "line" in its name?
line_idx <- which(grepl("line", names(smooth_grob$children)))
polyline_grob <- smooth_grob$children[line_idx][[1]]

line_grob <- segmentsGrob(0.7, 0.8,
                          grobX(polyline_grob, 45),
                          grobY(polyline_grob, 45),
                          arrow = arrow(angle = 10, type = "closed"),
                          gp = gpar(fill = "black"))

txt_grob <- textGrob("line of best fit", 0.71, 0.81, just = c("left", "bottom"))

gg_annotated <- grid::grobTree(
  gg,
  grid::grobTree(line_grob),
  grid::grobTree(txt_grob),
  vp = grid::viewport(x = unit(0.5, "npc"),
                      y = unit(0.5, "npc"),
                      width = unit(1, "npc"),
                      height = unit(1, "npc")
  )
)

grid.draw(gg_annotated)

The result:

If I adjust the y1 value for segmentsGrob by trial and error, e.g. by changing the grobY line to: grobY(polyline_grob, 45) + unit(0.045, "npc"), it produces the plot as I want it to look:

However, this is an arbitrary shift of the y-coordinate.

Perhaps I have retrieved the wrong grob. I have tried making the arrow end on other grobs in the grobTree but I have not been able to get this to work. How could I get the arrow to end on the line of best fit?