The need to delay retirement timing has been acknowledged in Western countries due to demographic ageing. The aim of the present study was to examine the buffering effects of job resources (decision authority, social support, work-time control, and rewards) on the association of exposures to physically demanding work tasks and physically hazardous work environment with non-disability retirement timing. Results from discrete-time event history analyses, in a sample of blue-collar workers (n = 1741; 2792 observations) from the nationwide longitudinal Swedish Longitudinal Occupational Survey of Health (SLOSH), supported that decision authority and social support may buffer the negative impact of heavy physical demands on working longer (continuing working vs retiring). Stratified analyses by gender showed that the buffering effect of decision authority remained statistically significant for men, while that of social support remained statistically significant for women. Moreover, an age effect was displayed, such that a buffering effect of social support on the association of heavy physical demands and high physical hazards with working longer were found among older men (≥64 years), but not younger (59–63 years). The findings suggest that heavy physical demands should be reduced, however, when not feasible physical demands should be accompanied by social support at work for delaying retirement.