New reference panel: mix precise/search-based data

Created by: mrnugget

The new reference panel currently only works with precise code intel. The code-intel-extensions reference panel falls back to search based code intel.

The following tasks are the result of the spike done in https://github.com/sourcegraph/sourcegraph/issues/31645 & https://github.com/sourcegraph/sourcegraph/pull/32823.

TODOs

• Basic fallback to search-based code intel when no precise code intel is available https://github.com/sourcegraph/sourcegraph/pull/32823
  • Determine which language spec to use for current document
  • Find token in text document, using language spec to determine token boundaries
  • Use language spec to determine file extensions
  • Query definitions
    • Filter definitions based on language spec's filterDefinitions
  • Query references
  • Tag results as search-based and display that in UI
  • Handle clicks on definitions in mini code view correctly
• Replicate behaviour of search-based code intel in code-intel-extensions
  • Complex query logic https://github.com/sourcegraph/sourcegraph/pull/33362
    • First search in current repository, at current commit
    • Then fallback to repositories that are not current repository, if not on Sourcegraph.com
    • Race between indexed/unindexed search
      • If basicCodeIntel.indexOnly is set, do not search unindexed
      • Use basicCodeIntel.unindexedSearchTimeout when searching unindexed search
  • Filter out isExternalPrivateSymbol (fix in symbols service layer)
  • Handle forks/archived repos, see this code comment https://github.com/sourcegraph/sourcegraph/pull/33277
    • Respect basicCodeIntel.includeForks
    • Respect basicCodeIntel.includeArchives
  • Sort results by proximity https://github.com/sourcegraph/sourcegraph/pull/33223
  • Handle codeIntel.mixPreciseAndSearchBasedReferences https://github.com/sourcegraph/sourcegraph/pull/33419
    • If there are no precise references: load search-based references
    • If there are precise references: only load search-based references if codeIntel.mixPreciseAndSearchBasedReferences is true. Filter out duplicates.