I'd go for Hardcore/Survival start changing mods like TSON Craft 5 or Better Beginnings (was going to choose but might not be for 1.12.2 and more so for 1.7.10 besides not finding it and would need to search around for it), but if I had to pick another for 1.12.2 or one I know is easy enough to link the source here, it would be Survivalist as those are the ones I remember commonly when it comes to changing up the game.
TSON Craft 5 (4 and under are Tech and Magic with the Hardcore survival elements but they have since been moved over to other mods and 5 is only the Hardcore survival elements now)
If you mean in a more mob focused way with like tougher mobs I can't think of many off the top of my head by name but I know they exist out there, someone might be able to recommend some hopefully when they see this thread.
Mobs, recipes and ability to access your crafting table/food, or a thirst system/stamina system added to give the player more to think about are the more common things I'd think of related to difficulty.
The Meaning of Life, the Universe, and Everything.
Ancient Warfare 2 adds some level of difficulty to the game. It adds structures you can raid that have all sorts of loot inside. But there are many powerful guards that stand in your path. Although this is a great mod, it doesn't add much difficulty once you are late game. For late game difficulty I recommend Draconic Evolution. Try to kill a chaos dragon, it's one of the most difficult bosses in modded minecraft