The frequency of severe droughts is increasing in many regions around the world as a result of climate change. Droughts alter the structure and function of forests. Site- and region-specific studies suggest that large trees, which play keystone roles in forests and can be disproportionately important to ecosystem carbon storage and hydrology, exhibit greater sensitivity to drought than small trees. Here, we synthesize data on tree growth and mortality collected during 40 drought events in forests worldwide to see whether this size-dependent sensitivity to drought holds more widely. We find that droughts consistently had a more detrimental impact on the growth and mortality rates of larger trees. Moreover, drought-related mortality increased with tree size in 65% of the droughts examined, especially when community-wide mortality was high or when bark beetles were present. The more pronounced drought sensitivity of larger trees could be underpinned by greater inherent vulnerability to hydraulic stress, the higher radiation and evaporative demand experienced by exposed crowns, and the tendency for bark beetles to preferentially attack larger trees. We suggest that future droughts will have a more detrimental impact on the growth and mortality of larger trees, potentially exacerbating feedbacks to climate change.