Realm keeps crashing

I'll be playing on the server with my friends and it'll occasionally crash and set us back about 20-30 minutes or when the last person saved and exited. Sometimes we can tell a crash is about to happen because people will move slower or it takes a second to load up the main menu or inventory screen. I have a 10 person limit on my realm but we currently only ever have 4 on at any one time. I switch between playing on Xbox and Nintendo Switch and several friends play on PC.