Right-clicking with sword still blocks

Dual-wielding doesn't work with swords, because when you right click with a sword it still blocks with it

The first screenshot is holding a sword (in hotbar/dominant hand) and a torch in the other slot. When I right-click it blocks with the sword.
The seconds screenshot is the same but holding an axe, and when I right-click it lets me place the torch.

This happens both in creative and survival