Villagers wont sleep and they constantly switch jobs

When it comes time for the villagers to go to sleep only a handful of them actually do. The beds aren't claimed by any other villager so I don't know any other cause than an issue with the game. Villagers will constantly lose their job and regain it causing a flicker effect. Hard to explain but i believe it is when they try and go for the same workstation and it causes interference.