Skip to content

Implement raw API

Created by: felixfbecker

We need a public HTTP endpoint to request a raw archive (zip, tar) for a repo, raw file contents and directory listings. Clients like language servers need to be able to resolve URLs relatively, e.g. construct a URL for a file given the URL for the directory and the file name.

Authentication must be embeddedable in the userinfo part of the URL so it can be encoded in an LSP URI:

          userinfo     host        port
          ┌─┴────┐ ┌────┴────────┐ ┌┴┐ 
  https://[email protected]:123/forum/questions/?tag=networking&order=newest#top
  └─┬─┘ └───────┬────────────────────┘└─┬─────────────┘└──┬───────────────────────┘└┬─┘  
  scheme     authority                 path              query                      fragment

The userinfo part persists when resolving relative URLs (the query does not).

URLs in the API must be unambiguous. Request for directories must be possible with a trailing slash, requests without a trailing slash should be rejected.

Content negotiation must happen through the Accept header (text/plain, application/zip, application/x-tar) and must not work by e.g. appending a different file extension like .zip or .tar to a URL, which would make URLs ambiguous. The API may also additionally support content negotiation through a query parameter like ?format=zip.

URLs for files must be sub-URLs of the URL to get an archive for the repo root (i.e. the repo root URL must be a strict prefix substring of the URL for any file in that repo). This is important so the repo root URL can be passed as a rootUri to language servers and file URLs are children of it.

The API must use HTTP status codes for errors, in particular

  • 404 Not Found if a file or directory does not exist
  • 401 Unauthorized if authentication is not valid (it must not redirect to a sign in page).
  • 406 Not Acceptable on an unsupported requested format

Errors should be returned in plain text by default, they should not be in HTML.

The API must support GET and HEAD requests. HEAD requests should not perform expensive retrieval of the resource contents, but return the correct status code so it can be used to determine for example if a file exists.

Directory listings must be encoded as a newline-delimited plain text list, it must not use JSON or some other structured format which would require parsing and inventing a schema.

Example URLs:

https://[email protected]/.api/raw/github.com/sourcegraph/sourcegraph@rev/
https://[email protected]/.api/raw/github.com/sourcegraph/sourcegraph@rev/web/
https://[email protected]/.api/raw/github.com/sourcegraph/sourcegraph@rev/web/gulpfile.ts