Holding ctrl or shift makes mouse 4 and 5 not work

I can't do anything with a reassigned action to mouse 4 and 5 while holding ctrl or shift.
For example pick block does not work with it reassigned to mouse 4 or 5(4th and 5th buttons on my mouse) while holding ctrl or shift.
It happens in singleplayer and multiplayer, survival and creative.