I made a repro codesandbox of the issue in action:
https://codesandbox.io/s/crazy-ully-9k98qs?file=/src/App.js
(you also change x to 7 at the top of the file to see the box correctly slide at first)
And in the other direction, sort of the opposite happens as well, where it gets stuck along the edge between polygons:
https://codesandbox.io/s/goofy-goodall-h27mny?file=/src/App.js
Thank you so much!