Yes, you are supposed to see multiple waves of the raids. If you have bad omen, you'll see a raid start, beat the first wave, and then see it restart with a second wave. Each is progressively harder until you beat all the waves and get the hero of the village effect. the villagers will toss you goodies, and you'll see fireworks when you are near them.
I would ring the bell so your villagers go into homes, and then lock them in with a block. Complete the raid until you get the hero of the village and it ends.
Your villagers *should* however all run into homes and hide.