Accurate surface heat flow data are required for a wide range of geological and geophysical applications. However, sediment temperature measurements beneath the seafloor often involve large uncertainties owing to the influence of bottom-water temperature (BWT) fluctuations. Previous studies reported apparently negative geothermal gradients in the Joetsu Basin of the Japan Sea and suggested that BWT fluctuations disturbed sediment temperatures. To address this problem, we monitored BWTs in the Joetsu Basin over a 2 year period to determine the depth at which the influence of BWT fluctuations on sediment temperature becomes negligible. Combined with sediment thermal diffusivity data, we determined that the BWT fluctuations can disturb sediment temperatures to a depth of 2 m. We obtained heat flow values of 81–88 mW m− 2 by measuring sediment temperatures at depths > 2 m using a 15 m long geothermal probe. The measured heat flow values are inversely correlated with topography owing to the effect of topographic change on the geothermal structure near the seafloor. A two-dimensional geothermal structure model was constructed to account for the topography, yielding an estimated regional background heat flow of 85 ± 6 mW m− 2. This study provides two important guidelines for obtaining accurate surface heat flow data in marine areas with large-amplitude BWT fluctuations: (1) quantitative information regarding BWT fluctuations and sediment thermal diffusivity is required to evaluate the depth range to which BWT fluctuations affect sediment temperature; and (2) information regarding the lithology and consolidation state of seafloor sediments is required for effective penetration using a long probe.