Skip to content

Search backend: convert closures and channels to stream

Warren Gifford requested to merge backend-integration/cc/event-handler into main

Created by: camdencheek

This modifies the stream-to-http code to fit into our normal streaming.Sender interface.

We jumped through a lot of hoops to convert a callback-based stream into a set of event channels, and it was honestly very confusing. This rewrites the code that handles converting events, populating metadata, and periodically flushing the results to the underlying stream into a type that implements streaming.Sender. This allows us to pass it directly into search.Execute and gets rid of a bunch of closures that implicitly capture a large amount of state.

Additionally, this provides a nice Done() method that cleans up resources, so I can more easily implement things like concurrent DB requests to populate repo metadata. This will be particularly useful for streaming pre-highlighted code.

Test plan

I audited this code carefully since time.AfterFunc can lead to some racy conditions. Depending on unit tests and backend integration tests to catch regressions, but also did some manual smoke tests.

Merge request reports

Loading