I have found that I need to put in a short delay (10 ms works for me) after the potentiometer read for this to work correctly.