Skip to content
Snippets Groups Projects
Closed symbol search slow at scale
  • View options
  • symbol search slow at scale

  • View options
  • Closed Issue created by Warren Gifford

    Created by: slimsag

    Because symbol search is implemented via repository -> SQLite regex matching, it can be very slow when hitting large repositories (100+ seconds for repositories on the order of ~20GB). This is in contrast to our zoekt search which is rather fast on a similar size repository I believe.

    Most of these perf issues occur in filterSymbols here: https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/cmd/symbols/internal/symbols/search.go#L155

    Possible solutions:

    1. Optimize simple regex matches to LIKE matches, e.g. f.*b.*r. becomes LIKE f%b%r_ to match foobar1. This is a good idea but will only help in simple cases.
    2. Prioritize proper indexing of symbols via zoekt. I believe this makes sense for multiple reasons:
      1. It would improve symbol search performance substantially in all cases
      2. It would improve perf of basic code intel and allow us to more easily support it on sourcegraph.com as both would be heavily reliant on symbol search perf.

    Reported by https://app.hubspot.com/contacts/2762526/company/407948923

    Activity

    • All activity
    • Comments only
    • History only
    • Newest first
    • Oldest first