Ladder is not effected by Efficiency Enchanted Axe

As mentioned in the title, when I tried to break ladders with my Efficiency IV axe. I tested it with an Efficiency V axe too, same problem.

I tried searching for keywords "axe", "ladder". None of them returned the same issue I am reporting right now, so if this is a duplicate please understand I didn't forget to search for similar problems.