crossbow_charge_time enchantment effect ignores slot field of enchantment

crossbow_charge_time enchantment effect works in both hands, ignoring slot field of enchantment. It is probably introduced by the fix of MC-272339.

Steps to reproduce

1. install the attached datapack and rejoin the world
2. take a crossbow and enchant it by /enchant @s test:never_charge
3. try to charge the crossbow in mainhand and offhand

Expected behavior
Since the enchantment is specified to work only in mainhand, the crossbow only takes very long time to charge in mainhand.
The charge time in offhand should be normal.

Observed behavior
The charge time is very long even if the crossbow is in offhand.