Soil moisture droughts have comprehensive implications for terrestrial ecosystems. Here we study time-accumulated impacts of the strongest observed droughts on vegetation. The results show that drought duration, the time during which surface soil moisture is below seasonal average, is a key diagnostic variable for predicting drought-integrated changes in (i) gross primary productivity, (ii) evapotranspiration, (iii) vegetation greenness, and (iv) crop yields. Drought-integrated anomalies in these vegetation-related variables scale linearly with drought duration with a slope depending on climate. In arid regions, the slope is steep such that vegetation drought response intensifies with drought duration, whereas in humid regions, it is small such that drought impacts on vegetation are weak even for long droughts. These emergent large-scale linearities are not well captured by state-of-the-art hydrological, land surface, and vegetation models. Overall, the linear relationship of drought duration versus vegetation response and crop yield reductions can serve as a model benchmark and support drought impact interpretation and prediction.