Raid bar disappears after a while in spectator mode

Steps to Reproduce:

1. Locate and teleport to the village
2. Give yourself the bad omen effect
3. Spawn some iron golems in the village
4. Go into spectator mode while the raid lasts

Observed Results:

The raid bar disappears after wave 2 or wave 3

Going back into creative also is known for causing the issue.

Expected Result:

The raid bar should not disappear and it should keep going.