Evidence from iron meteorites indicates that a large number of differentiated planetesimals formed early in Solar System history. These bodies should have had well-developed olivine-rich mantles and consequentially such materials ought to be abundant both as asteroids and meteorites, which they are not. To investigate this "Great Dunite Shortage" we have undertaken a geochemical and oxygen isotope study of main-group pallasites and dunitic rocks from mesosiderites. Oxygen isotope analysis of 24 main-group pallasites (103 replicates) yielded a mean Delta O-17 value of -0.187 +/- 0.016 parts per thousand (2 sigma), which is fully resolved from the HED Delta O-17 value of -0.246 +/- 0.014 (2 sigma) obtained in our earlier study and demonstrates that both groups represent distinct populations and were derived from separate parent bodies. Our results show no evidence for Delta O-17 bimodality within the main-group pallasites, as suggested by a number of previous studies. Olivine-rich materials from the Vaca Muerta, Mount Padbury and Lamont mesosiderites, and from two related dunites (NWA 2968 and NWA 3329), have Delta O-17 values within error of the mesosiderite average. This indicates that these olivine-rich materials are co-genetic with other mesosiderite clasts and are not fragments from an isotopically distinct pallasite-like impactor. Despite its extreme lithologic diversity the mesosiderite parent body was essentially homogeneous with respect to Delta O-17, a feature best explained by an early phase of large-scale melting (magma ocean), followed by prolonged igneous differentiation. Based on the results of magma ocean modeling studies, we infer that Mg-rich olivines in mesosiderites formed as cumulates in high-level chambers and do not represent samples of the underlying mantle. By analogy, recently documented Mg-rich olivines in howardites may have a similar origin. Although the Dawn mission did not detect mesosiderite-like material on Vesta, evidence linking the mesosiderites and HEDs includes: (i) their nearly identical oxygen isotope compositions; (ii) the presence in both of coarse-grained Mg-rich olivines; (iii) both have synchronous Lu-Hf and Mn-Cr ages; (iv) there are compositional similarities between the metal in both; and (v) mesosiderite-like material has been identified in a howardite breccia. The source of the mesosiderites remains an outstanding question in meteorite science. The underrepresentation of olivine-rich materials amongst both asteroids and meteorites results from a range of factors. However, evidence from pallasites and mesosiderites indicates that the most important reason for this olivine shortage lies in the early, catastrophic destruction of planetesimals in the terrestrial planet-forming region and the subsequent preferential loss of their olivine-rich mantles.