Smooth lighting causing low FPS and camera stuttering

Both options for "Smooth Lighting" Drops my framerate to 10 - 30 FPS. Setting it to "Off" removes the issue. This happens regardless of the "Fancy" or "Fast" Graphics setting. Standing still for a while will allow the FPS to recover until you start moving again.

With the Smooth Lighting enabled, there also seems to be an unusual jerky/stuttering movement with turning the camera, even when the FPS is at, or greater than 60.

Letting everything nearby load for a while helps stabilize the FPS. When Performance is set to Max FPS it gets the jerky camera. On Balanced and Powersaver the jerkiness mostly goes away.