Your issue is likely related to the way that you added the doors. I remember a video or description of this farm that mentioned that you need to layer the villages like you do in the Iron Titan. So, it's more complicated than just building it and putting 100 villagers in there.
The tutorial for the Iron Titan shows how multiple villages can be superimposed on top of each other. It's fiddly, but not overly complicated (well, a little complicated).
I would recommend watching the Iron Titan tutorial to understand how the layered village mechanics works: