The problem is, they're too intelligent. Dirt huts are more common then fortresses, many with basic defenses like the old "Dirt above air" door. Them randomly spawning is like meeting a zombie that is more efficient in every way (speed, strength, etc.). Be aware this means they will chase you when you're weaponless and still punching trees to begin a basic shack.
1
Caves are natural holes in the ground full of hideous monsters where you get ores, not Silent Hill: Geology Edition.
2
You posted this in the wrong topic, son.
1