Bacterial degradation of sinking diatom aggregates is key for the availability of organic matter in the deep-ocean. Yet, little is known about the impact of aggregate colonization by different bacterial taxa on organic carbon and nutrient cycling within aggregates. Here, we tracked the carbon (C) and nitrogen (N) transfer from the diatom Leptocylindrus danicus to different environmental bacterial groups using a combination of 13C and 15N isotope incubation (incubated for 72 h), CARD-FISH and nanoSIMS single-cell analysis. Pseudoalteromonas bacterial group was the first colonizing diatom-aggregates, succeeded by the Alteromonas group. Within aggregates, diatom-attached bacteria were considerably more enriched in 13C and 15N than non-attached bacteria. Isotopic mass balance budget indicates that both groups showed comparable levels of diatom C in their biomass, accounting for 19 ± 7% and 15 ± 11%, respectively. In contrast to C, bacteria of the Alteromonas groups showed significantly higher levels of N derived from diatoms (77 ± 28%) than Pseudoalteromonas (47 ± 17%), suggesting a competitive advantage for Alteromonas in the N-limiting environments of the deep-sea. Our results imply that bacterial succession within diatom aggregates may largely impact taxa-specific C and N uptake, which may have important consequences for the quantity and quality of organic matter exported to the deep ocean.