Diseases of marine animals caused by bacteria of the genus Vibrio are on the rise worldwide. Understanding the eco-evolutionary dynamics of these infectious agents is important for predicting and managing these diseases. Yet, compared to Vibrio infecting humans, knowledge of their role as animal pathogens is scarce. Here we ask how widespread is virulence among ecologically differentiated Vibrio populations, and what is the nature and frequency of virulence genes within these populations? We use a combination of population genomics and molecular genetics to assay hundreds of Vibrio strains for their virulence in the oyster Crassostrea gigas, a unique animal model that allows high-throughput infection assays. We show that within the diverse Splendidus clade, virulence represents an ancestral trait but has been lost from several populations. Two loci are necessary for virulence, the first being widely distributed across the Splendidus clade and consisting of an exported conserved protein (R5.7). The second is a MARTX toxin cluster, which only occurs within V. splendidus and is for the first time associated with virulence in marine invertebrates. Varying frequencies of both loci among populations indicate different selective pressures and alternative ecological roles, based on which we suggest strategies for epidemiological surveys.