Keep getting kicked out of my realm

Me and my friend regularly play on the realm together.

Usually late at night we both get kicked out every 15-20 minutes and any work done isn't saved

 

Meaning we have redo all our building work. Sometimes several times