Item frame can't be broken unless the block behind it is broken

What I expected to happen was:
When I left click an item frame in Creative, the item frame should break immediately

What actually happened was:
When I left clicked it nothing happened. I held down the left click and birch plank particles came out (birch plank was the block behind the item frame).
Then the birch planks broke and the item frame and the item in it came off and were items on the floor.