Current climate models are known to systematically overestimate the rate of deep water formation at high latitudes in response to too deep and too frequent deep convection events. We propose in this study to investigate a misrepresentation of deep convection in Hydrostatic Primitive Equation (HPE) ocean and climate models due to the lack of constraints on vertical dynamics. We discuss the potential of the Location Uncertainty (LU) stochastic representation of geophysical flow dynamics to help in the process of re-introducing some degree of non-hydrostatic physics in HPE models through a pressure correction method. We then test our ideas with idealized Large Eddy Simulations (LES) of buoyancy driven free convection with the CROCO modeling platform. Preliminary results at LES resolution exhibit a solution obtained with our Quasi-nonhydrostatic (Q-NH) model that tends toward the reference non-hydrostatic (NH) model. As compared to a pure hydrostatic setting, our Q-NH solution exhibits vertical convective plumes with larger horizontal structure, a better spatial organization and a reduced intensity of their associated vertical velocities. The simulated Mixed Layer Depth (MLD) deepening rate is however too slow in our Q-NH experiment as compared to the reference NH, a behaviour that opposes to that of hydrostatic experiments of producing too fast MLD deepening rate. These preliminary results are encouraging, and support future efforts in the direction of enriching coarse resolution, hydrostatic ocean and climate models with a stochastic representation of non-hydrostatic physics.