Include extensions integration tests into nightly builds
Created by: taras-yemets
Estimate the engineering effort and decide on the feasibility of adding some of the most popular extensions integration tests into the nightly builds.
One of the ways to implement it could be mocking user settings as if these extensions were enabled, fetching them from the extensions registry, and activating them (using the script loaded from the URL returned by extension registry response). Here's an example of using mocked extensions data.
With the suggested approach there are a few points for discussion:
- How do we define what to cover in these tests? Should tests include command palette, actions container, and decorations? Should we cover all the possible scenarios of extension settings changes and decoration types?
- It's likely that we're going to include extensions specific logic into
sourcegraph/sourcegraph
repo, are we comfortable with that? - Who is responsible for keeping the integration tests up to date? Should it be the extension PRs authors?
Any feedback and contribution are highly appreciated!