Populations with homogeneous distributions have better bet-hedging capacity than more heterogeneously distributed populations. Both population dynamics and environmental factors may influence the spatial variability of a population, but clear empirical evidence of such causal linkages is sparse. Using 25-year fish survey data from the North Sea, we quantify causal effects of age structure, abundance, and environment on nine fish species. We use empirical dynamic modeling—an approach based on state-space reconstruction rather than correlation—to demonstrate causal effects of those factors on population spatial variability. The causal effects are detected in most study species, though direction and strength vary. Specifically, truncated age structure elevates population spatial variability. Warming and spatially heterogeneous temperatures may enhance population spatial variability, whereas abundance and large-scale environmental effects are inconclusive. Fishing may affect population spatial variability directly or indirectly by altering age structure or abundance. We infer potential harmful effects of fishing and environmental changes on fish population stability, highlighting the importance of considering spatial dynamics in fisheries management.