Soil microorganisms living in close contact with minerals play key roles in the biogeochemical cycling of elements, soil formation, and plant nutrition. Yet, the composition of microbial communities inhabiting the mineralosphere (i.e., the soil surrounding minerals) is poorly understood. Here, we explored the composition of soil microbial communities associated with different types of minerals in various soil horizons. To this effect, a field experiment was set up in which mineral specimens of apatite, biotite, and oligoclase were buried in the organic, eluvial, and upper illuvial horizons of a podzol soil. After an incubation period of two years, the soil attached to the mineral surfaces was collected, and microbial communities were analyzed by means of Illumina MiSeq sequencing of the 16S (prokaryotic) and 18S (eukaryotic) ribosomal RNA genes. We found that both composition and diversity of bacterial, archaeal, and fungal communities varied across the different mineral surfaces, and that mineral type had a greater influence on structuring microbial assemblages than soil horizon. Thus, our findings emphasize the importance of mineral surfaces as ecological niches in soils.