Character will stare up at sky if left for 1 second

When character looks at certain blocks cursor auto moves so I am staring upwards at the sky and spinning after a 1 second count. Controller is correctly setup and works fine with other games and apps. I can stop the movement but it will start again as soon as I stop providing immediate input. Very annoying when try to dig as I am constantly adjusting my aim. Experimental Gameplay is enabled on one beta created world and not the other, problem persists on both. Seems to be connected with type of block I am looking at as wooden blocks,  ore blocks will cause it to shift, but not normal stone, dirt, moss covered stone, etc.