Skip to content

usagestats: use Amplitude HTTP API endpoint

Administrator requested to merge fa/amplitude-http into main

Created by: attfarhan

Currently, we use Amplitude for product analytics on Sourcegraph Cloud. Our current system uses event logs that are stored in our BigQuery data warehouse, and transforms these events for use in Amplitude, and then sends batches of events to the Amplitude Batch API using a python script.

We want to move to sending events to Amplitude directly from our application. This is the recommended way to instrument went using Amplitude, either through the SDK (client side) or HTTP API (server side). We opted to implement using the HTTP API. For more info see the RFC.

This PR implements sending events to Amplitude via the HTTP API. The main changes are summarized below.

GraphQL API: logEvent mutation

In the logEvent mutation, we now send a few new fields:

  • userProperties: user properties that we get from localStorage. For example, 'isSourcegraphTeammate' or 'hasAddedRepos'.
  • deviceID: required for Amplitude. A random ID assigned to the user's device.
  • eventID: used for deduplicating
  • insertID: used for deduplicating

Frontend event logger

Added getters for device ID, event ID, insert ID.

Webapp

In the SourcegraphWebApp component, we populate the user's localStorage with the latest user properties. We already have requests in the component to query the user's repositories, external services, and current authenticated user. This information is currently enough to get all the user properties we want. I'm wondering how this will scale if we ever need more info, so open to alternatives here.

Go usagestats package

We add a new codepath in the LogEvent function, where we call publishAmplitudeEvent if we're on Sourcegraph.com. publishAmplitudeEvent will get the Amplitude API key, and collect the arguments passed in with the event to create a JSON payload ready to send to Amplitude. This involves pulling different data to populate the userProperties field completely.

amplitude package

Then, we call the amplitude.Publish function. This function simply sends a POST request to the Amplitude HTTP API endpoint with the event.

This is the part I would like the most help on. I have some error handling and retry logic in there that could use some review. Also, is this simple implementation sufficient? I know a lot of Go client libraries for other tools get more complicated with concurrency, channels, etc, etc., but I haven't looked at that stuff for a long time and unsure if it's actually necessary.

Todo before merge:

  • Upload Amplitude API key.

Merge request reports