Can't break or place blocks unless camera is also moving with mouse/keyboard/OTG

I have connected the mouse and keyboard to my phone via OTG, and have noticed that if I try to dig or place blocks, with the camera stationary it doesn't work, however if the camera is moving it works. Please help