Cursor jumps while in inventory

Whenever I am in my inventory trying to move stuff around, my cursor will jump suddenly somewhere else. It only happens when using the left control stick. Makes it really hard to play when I can't get the cursor to do what I want it to do.