"io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection timed out: no further information" on Java Realms

I can play in my friends server for a good 5-10 minutes and then I disconnect with a message "connection lost internal exception java io ioexception". What I've heard it's a pretty common problem it never accrued to me until now.