Vibrios are frequently associated with oyster mortality; however whether they are the primary causative agent or secondary opportunistic colonizers is not well understood. Here we combine analysis of natural infection dynamics, population genomics and molecular genetics to ask (i) to what extent oysters are passively colonized by Vibrio population present in the surrounding water, (ii) how populations turn over during pathogenicity events and (iii) what genetic factors are responsible for pathogenicity. We identified several populations of Vibrio preferentially associated with oyster tissues. Among these, Vibrio crassostreae is particularly abundant in diseased animals while nearly absent in the surrounding water, and its pathogenicity is correlated with the presence of a large mobilizable plasmid. We further demonstrate that the plasmid is essential for killing but not necessary for survival in tissues of oysters. Our results suggest that V. crassostreae first differentiated into a benign oyster colonizer that was secondarily turned into a pathogen by introgression of a virulence plasmid into the population, possibly facilitated by elevated host density in farming areas.