Skip to content

httpapi: PoC of streaming API using SSE

Administrator requested to merge k/search-stream-backend-poc into main

Created by: keegancsmith

This is a proof of concept for the streaming API using server sent events. It is very minimal and doesn't send everything we need yet. For example it doesn't stream from the backend, misses out on important information (filters, stats), sends too much information for the normal use-case, etc. However, it is a good starting point for improvement and to develop the webapp against. This is based on an earlier fully featured prototype I did for streaming Zoekt which worked well.

SSE (Server-Sent Events) has been very nice to develop with. It is very well supported, simple to understand and simple to interact with. For example the simplest client for this API is as easy as this:

curl \
  -H 'Authorization: token REDACTED' \
  -H 'accept: text/event-stream' \
  'http://localhost:3080/.api/search/stream?q=my-query'

I intend for this API to be the same we use for communicating with backends such as Zoekt and Searcher. The frontend then may hydrate/enrich the data with extra information the webapp will find useful.

I'll create a later PR which documents the current protocol as it stands. But for people who develop against this right now I encourage just curling the endpoint and seeing what the response looks like. And then feel free to adjust it to suit the webapp's needs in particular.

We also mount this API on the UI endpoints. UI API allows requests from the webapp, while httpapi is useful for people accessing via an access token.

Part of https://github.com/sourcegraph/sourcegraph/issues/13067

Merge request reports