Indexed symbol search: definitive performance regression
Created by: slimsag
Workaround
When upgrading to Sourcegraph v3.7.1, set the following in your site configuration to avoid opting into the new symbol search implementation:
"search.index.symbols.enabled": false,Problem
After we introduced indexed symbol search in v3.7.1, both symbol and text search performance appears to have regressed. This issue is to investigate and fix that.
Note in the following that:
- All tests ran solely on their own for 17 minutes each (one test cannot affect the other).
- The name field indicates the exact search query that was being ran.
- The
P95 change(orange) column indicates the percentage change in 95th percentile query duration after migrating from Sourcegraph v3.5.2 to v3.7.1. Positive numbers are performance regressions, negative numbers are performance gains.
All queries were ran against theoa.sgdev.org over 10h period using vegeta and the CSV comparison report was generated using kingkai. The instance has 12,981 repositories In total cloned and indexed fully, and 42GB of data was generated by vegeta (happy to upload this somewhere if anyone is interested). I eliminated all external factors that I could not contribute to the code / design itself, i.e.:
- All repositories were completely cloned and reindexed.
- The "before" column actually indicates when Zoekt was constrained in CPU resources and Jaeger was enabled, the "after" column is actually after me doubling the CPU availability count and disabling Jaeger due to proven performance concerns with it. Given this, I expected to see substantially better perf, but did not.