Sourcegraph extension architecture is undocumented

Created by: tjkandala

Sourcegraph extension architecture is currently undocumented. Much of the thought and intention behind the current implementation is inside the heads of a couple of implementors, or has been lost to time.

We have some great high-level diagrams for our new architecture from RFC 155, but lower level diagrams and documentation would be very helpful for onboarding and debugging.

Proposals (in order of increasing implementation time + benefit)

1. Visual diagramming. This is straightforward to achieve with drag-and-drop flowchart software. The drawbacks are that there is no guarantee that the visual representation reflects actual behavior at runtime and visual diagrams are likely to fall out of date as we continue to work on the extension API. (possible tool: Lucidchart)

2. Text-to-diagram (generated diagrams). This is a bit more involved than visual diagramming, but it's more likely that we'd keep these up to date since we could review diffs alongside changes in application code. Unfortunately, this requires careful attention from developers, so there's still a chance that textual diagrams could become inaccurate. Also, there's still no guarantee that the generated diagrams reflect reality. (possible tool: GoJS)

3. Executable statecharts. From a single "source of truth", we could continuously generate diagrams which necessarily reflect the code's behavior at runtime. We would also be forced to explicitly enumerate our system's possible states, which could help prevent bugs (e.g. current race condition where extensions are sometimes activated after the current text document is emitted, so no actions are available). (possible tool: XState)