If Biome Depth weight is above ~1.12 you see these structures in the transition between ocean and deep ocean
If it's above ~1.99 you see them on the transition between a shore biome (Beach) and ocean.
Biome depth weight is a very useful control for altering features on a per-biome basis, but the range available without provoking serious weirdness (0-1.12) doesn't really do much.
I understand some value is going out of bounds and results in really tall parts in the transition between these biomes, because they use a lot of depth to join them together, could we at least get some bounding on it?
If they wanted to fix it they could - all they have to do is limit the range of a calculation, or a variable used in it, so the result can't go negative, which would have no effect on default and the majority of customized worlds (you can set BDW to more than 1.12 as long an increased biome depth offset prevents the calculation from giving negative values). If the result of the calculation is clamped (not BDW itself) it won't stop it from having any effect as BDW becomes higher since other biomes will still be affected; the bug just affects the biomes with the lowest base levels first (deep oceans, then oceans, then rivers, then swamps; any biome with a base level of 0 or more (above sea level) can never be affected). There is even a fix given in the bug report (I'd limit it to -1.999 instead of -1.8 to allow it to go as low as possible). They may have decided not to fix it because it would cause differences between new and old chunks in existing worlds but for the most part they would not be significant (besides the absence of spikes), or they consider the spikes to be a feature (but one that limits the usefulness of the setting. Maybe add a setting to control the result of the calculation?).
Also, it is worth noting that Mojang doesn't actually read (or own) these forums; even over on Reddit (which they do read) they probably wouldn't see it unless it got enough attention.