when farming crops, sometimes the farmland breaks

when I harvest crops, like wheat, carrots, of potatoes, sometimes the dirt block will break dropping the block, this also happened with nether warts on soulsand, it happens when you use your hand or any other tool, only about 5% of the time.