Create better retry mechanism for when provider errors out

[ykhli]

A few ideas

• set backup models to query from
• gracefully fail by generating an alternative from a local model