Raids end early with no indication why when raiders do not find a spawnable spot

Original summary
A raid ended after wave 1

We decided to build a small village on a nearby island in our realm, only has one villager so far. some pillagers spawned while I was nearby so I messaged my friends to do this raid with me, we all got really prepared for a hard battle. On wave of pillagers came... then we killed them. Then the raid bar just disappeared.