random fall damage

I fly around in my survival world and take 1-2 hearts of damage as if i have hit a block. And i could be building something, say five blocks high which you would normally take damage from, and not. but then i take damage as a “replacement” for that. (much more than i would have taken.)