Servers freezing mere seconds after logging in.

What happens is, everytime I log in to a server. I get to play normally as if nothing is wrong for about 10 seconds. After that every entity in the server just freezes and I get times out some time later. I haven't found out any issue like this before. People I asked around suggested that It could be caused by low internet speed but I really doubt that as I get around 30mbps from steam downloads. Any idea what causes this problem and how can I fix it ?