Search backend: convert closures and channels to stream
Created by: camdencheek
This modifies the stream-to-http code to fit into our normal streaming.Sender
interface.
We jumped through a lot of hoops to convert a callback-based stream into a set of event channels, and it was honestly very confusing. This rewrites the code that handles converting events, populating metadata, and periodically flushing the results to the underlying stream into a type that implements streaming.Sender
. This allows us to pass it directly into search.Execute
and gets rid of a bunch of closures that implicitly capture a large amount of state.
Additionally, this provides a nice Done()
method that cleans up resources, so I can more easily implement things like concurrent DB requests to populate repo metadata. This will be particularly useful for streaming pre-highlighted code.
Test plan
I audited this code carefully since time.AfterFunc
can lead to some racy conditions. Depending on unit tests and backend integration tests to catch regressions, but also did some manual smoke tests.