Piston turned on/stays on without redstone touching it

What I expected to happen was the piston (or sticky piston) would not turn on.

What actually happened was that the piston (or sticky piston) turned on when I had a block of redstone one block above it. When I tried it with a block to push it didn't move.

Redstone Block
Air
Piston

Steps to Reproduce:
1. Place down a piston facing sideways.
2. Place a block on top of the piston.
3. Place a redstone block on top of that block
4. Break the block in-between the piston and redstone block
5. The piston should extend
6. Break the redstone block
7. The piston should still be extended