Deep-sea microbial communities are exposed to high-pressure conditions, which has a variable impact on prokaryotes depending on whether they are piezophilic (that is, pressure-loving), piezotolerant or piezosensitive. While it has been suggested that elevated pressures lead to higher community-level metabolic rates, the response of these deep-sea microbial communities to the high-pressure conditions of the deep sea is poorly understood. Based on microbial activity measurements in the major oceanic basins using an in situ microbial incubator, we show that the bulk heterotrophic activity of prokaryotic communities becomes increasingly inhibited at higher hydrostatic pressure. At 4,000 m depth, the bulk heterotrophic prokaryotic activity under in situ hydrostatic pressure was about one-third of that measured in the same community at atmospheric pressure conditions. In the bathypelagic zone—between 1,000 and 4,000 m depth—~85% of the prokaryotic community was piezotolerant and ~5% of the prokaryotic community was piezophilic. Despite piezosensitive-like prokaryotes comprising only ~10% (mainly members of Bacteroidetes, Alteromonas) of the deep-sea prokaryotic community, the more than 100-fold metabolic activity increase of these piezosensitive prokaryotes upon depressurization leads to high apparent bulk metabolic activity. Overall, the heterotrophic prokaryotic activity in the deep sea is likely to be substantially lower than hitherto assumed, with major impacts on the oceanic carbon cycling.