Climate change is expected to profoundly affect key food production sectors, including fisheries and agriculture. However, the potential impacts of climate change on these sectors are rarely considered jointly, especially below national scales, which can mask substantial variability in how communities will be affected. Here, we combine socioeconomic surveys of 3,008 households and intersectoral multi-model simulation outputs to conduct a sub-national analysis of the potential impacts of climate change on fisheries and agriculture in 72 coastal communities across five Indo-Pacific countries (Indonesia, Madagascar, Papua New Guinea, Philippines, and Tanzania). Our study reveals three key findings: First, overall potential losses to fisheries are higher than potential losses to agriculture. Second, while most locations (> 2/3) will experience potential losses to both fisheries and agriculture simultaneously, climate change mitigation could reduce the proportion of places facing that double burden. Third, potential impacts are more likely in communities with lower socioeconomic status.