Fortune affects breaking tall flowers

Tall flower drops are affected by fortune waay too much. I'm getting 3-7 sunflowers/peony/rosebush/lilac etc. each time I break a stalk with a fortune 3 pickaxe, shovel, hoe or axe