Different mining speeds for nylium with fortune and silk touch

When mining any kind of nylium with a Fortune III, Efficiency V on a netherite pickaxe, it mines quickly but not insta-mine, but when mining the nylium with a Silk Touch, Efficiency V netherite pickaxe, the block is mined instantly.