Touching blocks will sometimes break in survival as if you were in creative

This happens more often if you are a client in a server. This can be reproduced more often if you are walking and then suddenly you press a block it wil break as if you were in creative.