Skip to content

Re-enable collecting search query terms

Created by: malomarrec

Background

We used to log search query terms on sourcegraph.com, to allow us to understand and optimise search results. As we prepared to roll-out search on private repositories, we deactivated collecting search queries: we collect some aggregated metrics, but we do not collect that users search for "foo bar".

The intention at the time was to deactivate collecting search queries terms only on private code, but for lack on bandwidth to rigorously implement two different approaches, all collection was disabled.

Problem

Disabling search query terms collection has made us half-blind to search issues on sourcegraph.com.

  • Un-authed users are difficult to optimise for, because besides analytics and some occasional widget feedback, we don't hear much from them.
  • User interviews are possible, but limited to a small sample which makes it easy to overlook core problems.
  • Aggregate analytics are great for tracking progress towards a known goal, but require careful planning and are not fit for post-hoc analysis, and detecting new trends in searches.

We care about optimising the experience of un-authed users because, while we believe signing-up and adding private code adds a lot of value to Sourcegraph, it should still be possible to search as an anonymous visitor and get great results.

Questions we care about:

  • are unauthed users utilizing the search query language? Are we setting them up for success?
  • do experiments around search learnability help new users onboard and affect query patterns?
  • are certain specific search queries not working in general?
  • as Sourcegraph is visited by more and more developers, do new search patterns emerge that we don't serve well?

Proposed solution

Re-enable search query terms collections, with the following strict boundaries:

  • we never collect search query terms when users are searching private code
  • we never collect search query terms when users are logged in
  • we carefully anonymise un-authed users when collecting search query terms