Component doesn't render properly when using lightning stylesheets

[madepally]

When I try to apply lightning stylesheets = true to the vf page that uses this component, the component doesn't render properly. The buttons to add and remove values are not functional.