Floating torches w/no blocks

If you put a redstone torch on the side of ANY block and destroy the block, the torch stays in mid air. This might not be a glitch, but I feel it's important to get this confirmed.