Describe the bug

When using a mouse (or a single pointer), zooming whilst panning causes the document to jitter as the pan start position is not adjusted.

Your environment

• Latest 7428761 (4.5.1)
• Google Chrome, version 115.0.5790.171

Expected behavior

No jitter when panning.

Actual behavior

Jitter is seen when panning and zooming. This jitter occurs after the first "mousemove" event after the zoom.

Steps to reproduce

1. Navigate to the "Panning and focal-point zooming" demo at https://timmywil.com/panzoom/demo/
2. Start a pan by clicking within the panzoom area (the further away from 0, 0, the more pronounced the effect will be)
3. Zoom using your scroll wheel
4. Pan the mouse again and observe a jitter as the object snaps to the wrong position.