Skip to content

Eliminate usage stats deprecated package

Warren Gifford requested to merge eliminate_usage_stats_deprecated_package-gh into main

Created by: dadlerj

Revisiting https://github.com/sourcegraph/sourcegraph/pull/38826

That PR had to be reverted due to slow performance and timeouts on the admin analytics page on dotcom. After testing locally with >30 million events in my local db, I could reproduce it (generating the admin chart took ~3 minutes).

The implementation relied on a SQL query that took ~20-30 seconds, which was called 9 times. I've slimmed it down to just one query that takes about the same amount of time (~30s).

First, I removed the integration user data—it was never actually used in any of the calls to this method.

Second, I changed the SQL to collect both registered and anon user data in a single pass.

Finally, I changed the SQL to aggregate users into DAU, WAU, and MAU buckets in a single pass as well.

I borrowed from the SiteUsage query (used for ping data aggregation), which is largely duplicative with this one, but is missing one key feature: It only allows you to pull the current DAUs, WAUs, and MAUs. If you want, e.g., "the last 3 months" of MAUs, you have to use this query/API call instead. I updated the naming of both functions to make it clear that they are related.

For reviewers, fell free to just look at the last four commits. The first six are identical to the first PR (just rebased).

Test plan

Tested locally

Merge request reports

Loading