So the 1.14 update is coming and villagers and villages are getting an overhaul, could maybe the villagers get an overhaul in their AI too so that they can be able to open iron doors with a button? Why or why not? helps.
As is (and without a pulse extender) villagers getting through the door while it is open probably requires more AI than can be afforded at present.
Pathfinding that allowed the use of ladders might be considerably easier (but would still represent a major change of course from only players being allowed these forms of interaction). [Starting with a greatly reduced tendency to wander into lava/off cliffs, or congregate until entity cramming 'thins the herd' seem both more doable and less disruptive.]
Rollback Post to RevisionRollBack
WARNING: I have an extemely "grindy" playstyle; YMMV — if this doesn't seem fun to you, mine what you can from it & bin the rest.