Skip to content

web: add experimental paginated search frontend

Warren Gifford requested to merge sg/experiment-paginated-search into main

Created by: slimsag

This PR

Adds an "experimentalFeatures": { "paginatedSearch": true } option to global/org/user settings. When true, it enables an experimental frontend which uses the paginated search API instead of the traditional search API.

The UI here simply queries results using the paginated API, and when the bottom of the virtual list is hit a loading indicator is displayed and more results are loaded (no "Show more" button).

Caveats

This will not be enabled by-default for the foreseeable future, because:

  1. The paginated search API currently does not handle non-text result types (diff,commit,symbol,etc).
  2. Most significant, the paginated search API is not optimized heavily yet and on e.g. sourcegraph.com in specific is extremely slow.
  3. The paginated search API makes a tradeoff for stable-ordered search results instead of fastest-first results, and we do not want to accept that tradeoff on customer instances by default.

Broader experiment

With caveats above noted, I would like to land this in order for me to perform broader experiments. This in particular is part of a broader experiment I am making in which I think the following might be a good idea on Sourcegraph.com specifically:

  1. Paginated search on-by-default
  2. A very large cluster of unindexed searchers (e.g. a few hundred or maybe even thousand, exact number TBD)
  3. Ability for customers to easily declare "search in my org's 1-20k repos by default"

The theory here is that we could support a large number of enterprise customers with a large cluster of unindexed searchers without having to scale linearly using indexed search, reducing costs for us and customers while still providing a very nice nearly instant search across hundreds or thousands of repositories. This would be possible on Sourcegraph.com only due to the economies of scale savings effect we could have (unindexed searchers shared across all customers effectively).

Merge request reports