Skip to content

codeintel: ingest, process, and expose API documentation data

Warren Gifford requested to merge sg/lsif-documentation into main

Created by: slimsag

This PR makes the LSIF backend ingest, process, and expose API documentation data over GraphQL.

  • It ingests the data produced in the format described by the Sourcegraph API documentation extension, i.e. what is implemented in https://github.com/sourcegraph/lsif-go/pull/150.
  • For storage, we store gob-encoded payloads in the DB at a per-page granularity. That is, while the extension describes hierarchy of documentation at a somewhat arbitrary granularity, the DB stores at a per-page granularity. Clients can then query pages using a path ID which is a path made up of the sum of all documentationResult slugs pointing to that page.
  • The exposed GraphQL API supports per-page querying, you can query a single page or start at the root / and walk the pages below that. Since GraphQL has terrible support for recursive data structures, we transfer an opaque JSON payload instead.

Below is an example of the type of GraphQL query which works after this change, querying the documentation pages starting at the root of the project.

image

Fixes https://github.com/sourcegraph/sourcegraph/issues/19592 Fixes https://github.com/sourcegraph/sourcegraph/issues/19593 Helps https://github.com/sourcegraph/sourcegraph/issues/19389

Merge request reports