endpoint: recover from errors in discovery

Created by: keegancsmith

If an error is returned by discovery we never recover from it. Currently a customer is working around this by restarting our frontend. This can happen frequently at smaller scales if a roll out temporary takes down all endpoints in a service.

Test Plan: Added a unit test which exercised the bug. The test failed until the fix.