An on-farm field experiment on a locally adapted conservation tillage method was undertaken to evaluate its effect on soil erosion, surface runoff, and agronomic parameters. It was conducted on five farmer fields with 3–14% slopes in the Rift Valley and the Eastern escarpment of Ethiopia’s central highlands region for two cropping seasons. The treatments were conventional tillage (CT), repeated ploughing performed with a traditional ox-drawn plough named ‘Maresha’, and minimized contour ploughing (MT) at most twice with a locally adapted sweep-like attachment assembled to Maresha. Surface runoff and soil loss in the MT system were 30 to 60% and 49 to 76% lower than those in the CT system on 3 to 14% slopes, respectively. Despite the wide variation in surface runoff, limited differences in soil water content for the depth from 0 to 20 cm were observed between the treatments. Significant differences (p < 0.05) in grain yields (kg ha−1) of 246 and 323 in the 1st and 2nd growing seasons, respectively, were recorded between the MT and CT treatments. The results of this study demonstrated that the MT system can significantly reduce surface runoff and soil loss while improving crop yields in rainfed smallholder farming systems of Ethiopia.