Redstone Circuits stay on when they are not powered.

My redstone circuit stayed on when it's not powered.

What I expected to happen was...:
That it would turn off when it was not powered.

What actually happened was...:
Part of the redstone circuit stayed on when it was not powered.

Steps to Reproduce:
1. Two images of the redstone circuit can be seen in the attachment.