cannot mine ore

I made a diamond pickaxe and put a couple enchants on it, one of them was the silk touch. I used it for about an hour, then switched back to my efficiency III and unbreaking II because I wasn't getting ore, just blocks.. I am still only getting blocks, only with metal. works fine with coal, redstone, lapis.