Redstone torches

If i place a sandstone block (in this case, didnt check others yet) and i put a redstone torch on it, its all fine. but if i remove the block the redstone torch will stay floating on the exact same place where i put it.