The interplay among microorganisms profoundly impacts biogeochemical cycles in the ocean. Culture‐based work has illustrated the diversity of diatom–prokaryote interactions, but the question of whether these associations can affect the spatial distribution of microbial communities is open. Here, we investigated the relationship between assemblages of diatoms and of heterotrophic prokaryotes in surface waters of the Indian sector of the Southern Ocean in early spring. The community composition of diatoms and that of total and active prokaryotes were different among the major ocean zones investigated. We found significant relationships between compositional changes of diatoms and of prokaryotes. In contrast, spatial changes in the prokaryotic community composition were not related to geographic distance and to environmental parameters when the effect of diatoms was accounted for. Diatoms explained 30% of the variance in both the total and the active prokaryotic community composition in early spring in the Southern Ocean. Using co‐occurrence analyses, we identified a large number of highly significant correlations between abundant diatom species and prokaryotic taxa. Our results show that key diatom species of the Southern Ocean are each associated with a distinct prokaryotic community, suggesting that diatom assemblages contribute to shaping the habitat type for heterotrophic prokaryotes.