A low intake of fruit and vegetables is a significant contributor to the global burden of disease. The aim of this study was to estimate the size of the risk of type 2 diabetes (T2D) of a low intake and to investigate possible sex differences. In this regard, this study used a longitudinal data from the Stockholm Public Health Cohort located in Sweden, collected in 2010 and 2014. The analysis included 14 718 men and 20 589 women aged 25 to 84 years. Fruit and vegetable intake, separately <2 servings/d or combined <4 servings/d (one serving corresponding to 100 g) was set as a cut-point for low intake. The sex difference at baseline was examined. Sex-stratified logistic regression was performed with onset of T2D as the outcome and fruit and vegetable intake at baseline as the exposure with adjustment for other known risk factors. Results indicate that men consumed significantly (P < 0.001) less fruit and vegetables compared with women. A 62 % higher risk to develop T2D over the 4-year period was observed in men who had low vegetable intake compared with high intake after adjusting for age, education, BMI, smoking, alcohol and physical activity (OR 1.62; 95 % CI 1.00, 2.63). In women, a significantly higher risk of T2D was also observed with a low intake of vegetables, but not after adjustment. The present study suggests that higher consumption of vegetables seems to be protective for the onset of T2D in men. Thus, increasing the intake of vegetables in men should be a public health priority.