Skip to content

[WIP] Codeinsights MVP backend

Warren Gifford requested to merge codeinsights into main

Created by: slimsag

See #17218 (closed) for the general plan / work being done on code insights.

I am working from this codeinsights branch (which will be unstable/buggy/broken sometimes) and my aim is to have this branch by EOW run a very basic code insights backend in dev environments, including all the app infra/changes we need, running timescaledb, doing the DB migrations to create the schema, and the GraphQL schema/API/backend making requests and serving data from timescale.

This branch itself won't be merged, rather I will be taking changes from here once they are solid and sending individual PRs for review (so they meet our quality bar, are not "one big PR", etc.) I will be posting updates here as I send out those PRs.

TODO:

  • Grafana dashboards -- issue filed elsewhere
  • testing: An end-to-end test where we use the GraphQL API to modify user settings/add insights, and the GraphQL API to query data after a while? -- issue filed elsewhere
  • Investigate adding more tests for enterprise/internal/insights/background/* - see https://github.com/sourcegraph/sourcegraph/pull/18267 -- issue filed elsewhere
  • Basic webhook fetching support -- decided to adopt push model, see other issue
  • Deploy TimescaleDB in non-dev envs
  • Generate schema.insights.md documentation
  • dev/db/squash_migrations.sh starts Postgres 9.6 if the connected DB version != 9.6, so it doesn't work with these migrations.
  • internal/db/schemadoc/main.go starts Postgres 9.6 if the connected DB version != 9.6, so it doesn't work with this DB schema yet.
  • Add code TODO for retroactively updating repo names in DB

Feb 21-26:

  • insights: correct over-reporting / aggregation of data points #18632

Feb 15-19:

Feb 8-9:

  • Implement DB store tests
  • Merge DB schema
  • Implement GraphQL backend tests

Feb 1-5:

Jan 18-22:

  • Determine where to store insights (user/org/global settings? likely not DB for now, but file an issue for later?)
  • Run migrations on server startup
  • Plan DB schema & queries/inserts
  • Generate separate schema.md files for codeintel/frontend DBs so we can generate schema.insights.md later
  • Tag TimescaleDB Docker image as sourcegraph/codeinsights-db
  • Run codeinsights-db as part of dev server.
  • Add migrations/codeinsights DB migration foundation
  • Add GraphQL backend scaffolding / stubs

Merge request reports

Loading