Biological nitrogen fixation contributes significantly to marine primary productivity. The current view depicts few cyanobacterial diazotrophs as the main marine nitrogen fixers. Here, we used 891 Tara Oceans metagenomes derived from surface waters of five oceans and two seas to generate a manually curated genomic database corresponding to free-living, filamentous, colony-forming, particle-attached, and symbiotic bacterial and archaeal populations. The database provides the genomic content of eight cyanobacterial diazotrophs including a newly discovered population related to known heterocystous symbionts of diatoms, as well as 40 heterotrophic bacterial diazotrophs that considerably expand the known diversity of abundant marine nitrogen fixers. These 48 populations encapsulate 92% of metagenomic signal for known nifH genes in the sunlit ocean, suggesting that the genomic characterization of the most abundant marine diazotrophs may be nearing completion. Newly identified heterotrophic bacterial diazotrophs are widespread, express their nifH genes in situ, and also occur in large planktonic size fractions where they might form aggregates that provide the low-oxygen microenvironments required for nitrogen fixation. Critically, we found heterotrophic bacterial diazotrophs to be more abundant than cyanobacterial diazotrophs in most metagenomes from the open oceans and seas, emphasizing the importance of a wide range of heterotrophic populations in the marine nitrogen balance.