httpapi: PoC of streaming API using SSE

Created by: keegancsmith

This is a proof of concept for the streaming API using server sent events. It is very minimal and doesn't send everything we need yet. For example it doesn't stream from the backend, misses out on important information (filters, stats), sends too much information for the normal use-case, etc. However, it is a good starting point for improvement and to develop the webapp against. This is based on an earlier fully featured prototype I did for streaming Zoekt which worked well.

SSE (Server-Sent Events) has been very nice to develop with. It is very well supported, simple to understand and simple to interact with. For example the simplest client for this API is as easy as this:

curl \
  -H 'Authorization: token REDACTED' \
  -H 'accept: text/event-stream' \
  'http://localhost:3080/.api/search/stream?q=my-query'

I intend for this API to be the same we use for communicating with backends such as Zoekt and Searcher. The frontend then may hydrate/enrich the data with extra information the webapp will find useful.

I'll create a later PR which documents the current protocol as it stands. But for people who develop against this right now I encourage just curling the endpoint and seeing what the response looks like. And then feel free to adjust it to suit the webapp's needs in particular.

We also mount this API on the UI endpoints. UI API allows requests from the webapp, while httpapi is useful for people accessing via an access token.

Part of https://github.com/sourcegraph/sourcegraph/issues/13067

Created by: keegancsmith

Did this PoC 3 weeks ago, should of just merged it in then instead of developing against it in private. Lesson learnt :)

Created by: keegancsmith

@stefanhengl aah missed that you just reviewed this. I just force pushed a change which extracts the search.go into a package, and then mounts that handler on two endpoints. So it is now available at both .api/search/stream and search/stream. The reason we do both is because src-cli/etc will access this via .api, while the UI will access via search/stream. I couldn't work out (yet) how to make server sent events work via the .api url. We do manage to do graphql requests against .api from the webapp though. So this will be something I change later.