Skip to content

search: mint search routine type

Administrator requested to merge main-dry-run/rvt/mint-routine into main

Created by: rvantonder

This mints a Routine type to hold the 3-tuple values previously used to evaluate a toplevel search:

type Routine struct {
	Jobs        []Job
	RepoOptions search.RepoOptions
	Timeout     time.Duration
}

After thinking about a name for this type and settling on Routine, and propagating it, I think it works well. See docstring for more details.

This is semantics-preserving. Comments and functions are updated to reflect this type. This structure is generally more comprehensible, but there's another motivation. Optional reading below if you want to understand why.

The main motivation is that I want to move from our current evaluate*(...) functions that operate on query.Basic, to instead evaluate on run.Routine. That is, a search query plan gets converted to a tree of run.Routine. This conversion phase is needed if we want to do optimizations (like Zoekt and) cleanly. Currently there's an issue with the order in which search jobs/routines are evaluated. If we implemented the Zoekt optimization now, we'd effectively have to go and find search routines containing zoekt jobs (where the work has already been split up) and replace it with the Zoekt-optimized job, OR, we'd have to add the Zoeekt-optimized job to a routine, and then pass some conditional so that subsequent and logic does not also add unoptimized Zoekt jobs. If this doesn't quite make sense, don't worry about it. The point is that the eval functions operate on a query.Basic type and impose a certain ordering of job/routine creation that makes it difficult to modify the output of that creation for optimized backend queries. The solution is to not operate on query.Basic during eval, but instead convert it before eval to a tree of routines. The intent has always been to not evaluate on query.Basic and I made a comment about this ages ago--we want to evaluate on our internal type (best represented by Routine now).

There is no reason we can't turn a query into a tree of Routines, and, in fact, our type Plan []query.Basic is going to eventually be type Plan <tree of routines>. Actually that will probably happen pretty soon, because that representation is conducive to the optimization phase. Keegan has brought up the idea of an "IR" for our search code, and type Plan <tree of routines> is currently our best representation of that "IR". Notably that representation of a backend plan should not (and should never have had to) care directly about anything in the query string.

Merge request reports