Skip to content

dev/sg: track local sg analytics that can be submitted manually

Administrator requested to merge sg-okay-analytics into main

Created by: bobheadxi

Extends sg such that:

  1. Each run of sg will now aggregate events in context via analytics.LogEvent calls
  2. Each run of sg now logs an overall command duration event with analytics.LogEvent, with optional events that happened (error, cancellation, success)
  3. As a demonstration, log time-to-install for each command in sg start and time to check for updates in the auto-updater
  4. At the end of each run of sg, aggregated events will be stored to disk (~/.sourcegraph/events), not submitted to OkayHQ, and will attach command context (command names) and the names of flags that are used. We do not log arguments and values that could be sensitive.
  5. Analytics collection can be disabled with --disable-analytics or SG_DISABLE_ANALYTICS

Introduces the following commands:

  • sg analytics view: List all analytics events currently on disk.
  • sg analytics reset: Delete all analytics events currently on disk.
  • sg analytics submit: Submit all analytics currently on disk to OkayHQ using OKAYHQ_TOKEN or CI_OKAYHQ_TOKEN from gcloud (we can change this but for now this seems most convenient)

This PR is scoped to duration events. We can perhaps extend this to track errors as well in a follow-up.

Currently, we do not automatically submit these events, and I would prefer that to be enabled in a follow-up.

Kicks off https://github.com/sourcegraph/sourcegraph/issues/33447

API usage

sg start install success:

analytics.LogEvent(ctx, "install_command", []string{cmdName}, time.Since(installationStart), "succeeded")

sg start install fail:

analytics.LogEvent(ctx, "install_command", []string{failure.cmdName}, time.Since(installationStart), "failed")

Schema

See sg analytics view --raw

Action event:

    {                                                                                                                 
      "Name": "sg_action",                                                                                            
      "Timestamp": "2022-05-06T05:00:07.352442Z",                                                                     
      "Metrics": {                                                                                                    
        "cancelled": {                                                                                                
          "type": "count",                                                                                            
          "value": 1                                                                                                  
        },                                                                                                            
        "duration": {                                                                                                 
          "type": "durationMs",                                                                                       
          "value": 3638                                                                                               
        }                                                                                                             
      },                                                                                                              
      "Labels": [                                                                                                     
        "sg",                                                                                                         
        "start"                                                                                                       
      ],                                                                                                              
      "UniqueKey": [                                                                                                  
        "event_id",                                                                                                   
        "context",                                                                                                    
        "event_name",                                                                                                 
        "event_version",                                                                                              
        "run_id"                                                                                                      
      ],                                                                                                              
      "Properties": {                                                                                                 
        "command": "sg start",                                                                                        
        "context": "sg",                                                                                              
        "event_id": "565d6044-a7d7-4ee6-9ba1-b84c2a98c135",                                                           
        "event_name": "sg_action",                                                                                    
        "event_version": "v0",                                                                                        
        "flags_used": "",                                                                                             
        "run_id": "0c7c66ac-b264-453d-b9b1-39a70b1a3d3e",                                                             
        "sg_version": "dev"                                                                                           
      }                                                                                                               
    }   

Command install event:

    {                                                                                                                 
      "Name": "install_command",                                                                                      
      "Timestamp": "2022-05-06T05:00:07.803753Z",                                                                     
      "Metrics": {                                                                                                    
        "duration": {                                                                                                 
          "type": "durationMs",                                                                                       
          "value": 3129                                                                                               
        },                                                                                                            
        "succeeded": {                                                                                                
          "type": "count",                                                                                            
          "value": 1                                                                                                  
        }                                                                                                             
      },                                                                                                              
      "Labels": [                                                                                                     
        "zoekt-webserver-0"                                                                                           
      ],                                                                                                              
      "UniqueKey": [                                                                                                  
        "event_id",                                                                                                   
        "context",                                                                                                    
        "event_name",                                                                                                 
        "event_version",                                                                                              
        "run_id"                                                                                                      
      ],                                                                                                              
      "Properties": {                                                                                                 
        "command": "sg start",                                                                                        
        "context": "sg",                                                                                              
        "event_id": "24a2f245-6966-482a-8a0a-7335c1bc3214",                                                           
        "event_name": "install_command",                                                                              
        "event_version": "v0",                                                                                        
        "flags_used": "",                                                                                             
        "run_id": "0c7c66ac-b264-453d-b9b1-39a70b1a3d3e",                                                             
        "sg_version": "dev"                                                                                           
      }                                                                                                               
    } 

Test plan

image

I don't actually see the events in OkayHQ at the moment despite the API telling me I'm good. I am going to schedule a time with an OkayHQ rep to chat about events next week

Merge request reports

Loading