The Southern Ocean south of 35 degrees S accounts for approximately half of the annual oceanic carbon uptake, thereby substantially mitigating the effects of anthropogenic carbon dioxide (CO2) emissions. The intensity of this important carbon sink varies considerably on inter-annual to decadal timescales. However, the drivers of this variability are still debated, challenging our ability to accurately predict the future role of the Southern Ocean in absorbing atmospheric carbon. Analysing mapped sea-air CO2 fluxes, estimated from upscaled surface ocean CO2 measurements, we find that the overall Southern Ocean carbon sink has weakened since similar to 2011, reversing the trend of the reinvigoration period of the 2000s. Although we find significant regional positive and negative responses of the Southern Ocean carbon uptake to changes in the Southern Annular Mode (SAM) over the past 35 years, the net effect of the SAM on the Southern Ocean carbon sink variability is approximately zero, due to the opposing effects of enhanced outgassing in upwelling regions and enhanced carbon uptake elsewhere. Instead, regional shifts in sea level pressure, linked to zonal wavenumber 3 (ZW3) and related changes in surface winds substantially contribute to the inter-annual to decadal variability of the Southern Ocean carbon sink.