There does not seem to be any way to clear idle connections from servers that were removed from ServerList. Assuming I have ServerList implementation that allows adding/removing servers (for example for AWS ElastiCache) and I remove some server, the free connections map will keep those connections forever.
Ideally there should be something like dropAllIdle() that would cause dropping of all idle connections. That should be simple to implement and at the same time good enough.