Blocks frequently reappear after having been mined

I usually play on a minefold.com server, and I've noted that the latency is not always that great. VERY often, when I mine blocks, they will reappear a few seconds later. Sometimes after another few seconds they will disappear. Sometimes I have to mine them again to really get rid of them.

I am pretty sure this has to do with a high latency connection to the server because I've noticed it gets worse when my connection has higher latency.