CAnnot connect to Realm during evening

When trying to connect to a friend's realm during afternoon and evening I connect only to fall into the void with the world loading in above me. After about 10 seconds of falling I am disconnected with the error: internal exception java.net.socketexception connection reset.

I have checked my ping and firewall setting and these are fine. When falling into the void I can see other players online by holding tab, and they see my model standing still, but i cannot see chat messages, etc. 

This does not happen in the morning, when I am able to connect fine.