Skip to content

usage-data: push scraped events to pubsub if telemetry job is enabled

Warren Gifford requested to merge usage-data/push-to-pubsub into main

Created by: coury-clark

Closes https://github.com/sourcegraph/sourcegraph/issues/39088

  1. Push events that are scraped to pubsub.
  2. Adds site settings to control the project / topic

Note: If you inspect the pubsub topic below you will see a variety of message formats. There is still some uncertainty in the format, so for now we won't worry so much that the schema is exactly correct in the destination.

Test plan

To run / test locally:

Using credentials: https://start.1password.com/open/i?a=HEDEDSLHPBFGRBTKAKJWE23XX4&h=team-sourcegraph.1password.com&i=fpusirhqcoi744acjtowvzfixq&v=dnrhbauihkhjs5ag6vszsme45a

Copy that file to a location, and then start sg with

GOOGLE_APPLICATION_CREDENTIALS="path/to/file.json" sg start...

Add to your site config

  "exportUsageTelemetry": {
    "enabled": true,
    "topicProjectName": "sourcegraph-dogfood",
    "topicName": "usage-data-testing"
  },

A variety of events can be found here: https://console.cloud.google.com/cloudpubsub/topic/detail/usage-data-testing?authuser=0&project=sourcegraph-dogfood&tab=messages from pushing locally.

Merge request reports