Skip to content

Publish automation and code intel usage statistics to BigQuery.

Administrator requested to merge publish-automation-code-intel-usage into master

Created by: efritz

This PR currently subsumes https://github.com/sourcegraph/sourcegraph/pull/8374, but will merge with master to collapse the changes.

This PR publishes the new pings to BigQuery. Previous PRs (including the one above) have already added this data in the updatecheck from private instances to the DotCom instance. This change will require the BigQuery schema and the Dataflow transform to be modified once this change is deployed.

@ebrodymoore, I will need your help modifying the transform, the schema, and the ETL pipeline. This does not need to occur before the branch cut, but if you are available today it may be a good time to work on it.

Automation fields:

automation_usage: NULL or {
	CampaignsCount
}

Code intel fields:

code_intel_usage: NULL or {
		Daily: array of {
			StartTime: date string
			Hover: {
				LSIF: {
					UsersCount: int
					EventsCount: int or null
					EventLatencies: {
						P50: float
						P90: float
						P99: float
					}
				}
				LSP: (see above)
				Search: (see above)
			},
			Definitions: (see above),
			References: (see above)
		},
		Weekly: (see above),
		Monthly: (see above)
	}
}

For sanity, the site usage fields:

site_activity {
	NULL or {
		DAUs {
			StartTime
			UserCount
			RegisteredUserCount
			AnonymousUserCount
			IntegrationUserCount
			Stages NULL or {
				mng, plan, code, rev, ver, pkg, depl, conf, mtr, sec, auto (all ints)
			}
		}[]
		
		WAUs {
			(see above)
		}[]
		
		MAUs {
			(see above)
		}[]
	}
}

Merge request reports

Loading