I've spend just a little time in the End, and I've encountered similar problems. The bug with Spectre Key and End dimension is real, Ender Dragons are everywhere and they aren't threat anymore.
However I had problem with Endermen, exactly the Angry ones, even with best armor (Fabulous, Dark, etc.) and 60HP they killed me in 2-3 hits. Fighting the EnderEye was just impossible, whenever I landed on the tower, there started to rain AngryEndermen everywhere which results in instant kill, so I made it only with SPAMR from reasonable distance. I felt just strange that EnderDragon hits you (in armor) for 1-2HP while Enderman for 25-30HP
After you beat the game, you can go back to any dimension, it's the purpose of the last quest. If you want to continue the Journey, you should try the Dawn of the Void modpack, which poses as sequel to this pack.
I've never had problems tanking angry endermen. My armor was comprised of a Soularium Jetplate and 3 pieces of Dark Steel armor with Protection IV, three levels of empowerment, and Soulbound, nothing hit me for more than a heart of damage at a time and my cleaver had Life Steal III so the end was a breeze for me. Though my methodology for attacking the eye would be to clear the tower out first, from the bottom breaking all the spawners on the way up -- which is time-consuming but safe. I gave up on doing the rest of the hardcore ender expansion because I couldn't stand all the dragons and the slow chunkloading.
Thanks for the mod recommendation.