Skip to content

Search backend: pass stream through explicitly

Warren Gifford requested to merge main-dry-run/cc/always-stream into main

Created by: camdencheek

This modifies our search pipeline to pass the stream through each and every step that needs it. The goal here is to remove references to r.stream, especially mutations of r.stream so that we can be sure that either 1) we are running in streaming mode and stream is non-nil, or 2) we are running in batch mode and stream is nil.

From here, we can modify evaluateAnd and evaluateOr to be compatible with streaming. They will still aggregate their results (because results from the same file need to be merged), but the aggregated results will be sent up the parent stream if it is non-nil.

Once we get past that step, we can pass in an aggregating stream when running resultsBatch so that stream is always non-nil. With that guarantee, we unlock a bunch of other simplifications.

Stacked on #30454

Merge request reports

Loading