web: add web-app server for development and production builds

Created by: valerybugakov

Independent web app server

This PR introduces an independent server for the web application which can use any deployed API instance as a backend.

Context

While working on the CSS modules configuration, I needed a way to iterate quickly on the webpack.config.js and test changes in the production environment: run yarn build-web and check how CSS modules are loaded in the application. There was no easy/quick way to do it. @umpox found out that it's possible to change a couple of variables in the start.sh script to serve the production build. But this solution was quite slow because of the backend services involved.

To solve this, I've configured a minimum viable server to serve web app production artifacts without our backend services. I've added HtmlWebpackPlugin to our webpack.config.js to produce index.html and configured Webpack dev server proxy to point the server to the deployed API instance. It allowed testing changes in production build without backend services quickly. This PR is the enhanced version of this setup.

How it works

1. HtmlWebpackPlugin is added to the Webpack config. It can be enabled by the env variable WEBPACK_SERVE_INDEX=true. This plugin outputs index.html to the ui/assets.
2. Express server is used for a production build. Webpack-dev-server in the development environment.
3. Server proxies API requests to SOURCEGRAPH_API_URL provided in the .env file using the http-proxy-middleware.
4. To avoid the CSRF token is invalid API error CSRF token is retrieved from the SOURCEGRAPH_API_URL before the server starts. Then this value is used for every subsequent request to the API.
5. To enable authentication via the sgs cookie, all cookies from the API requests are propagated to the local client.

Known issues

The issues listed below can be addressed in separate PRs. They are non-blocking for the functionality this PR introduces and doesn't affect existing workflows. We can iterate on each individual issue once we have the foundation in place.

1. HTTPS support is not in place yet, because of (https://github.com/webpack/webpack-dev-server/issues/2313). It needs more investigation.
2. A static mock in combination with a local site-config.json is used instead of the JS context returned from the SOURCEGRAPH_API_URL.
3. Authentication providers don't work. This is probably related to the social login links, which are served along with JS context.
4. A couple of variables are duplicated from gulpfile.js. They should be shared with it after migration to Typescript.
5. index.html template is different from what we have in cmd/frontend/internal/app/ui/app.html. We should reuse it to be consistent with the default production setup.

Changes

1. Two commands are added to the client/web/package.json: yarn serve:dev and yarn serve:prod.
2. dev directory is added to the client/web package. It contains the server configuration used by the commands above.
3. .env.example is added to the client/web to showcase the example env configuration.
4. require('ts-node').register() is added to the gulpfile.js to allow TS usage for html-webpack-plugin configuration.

Updates

1. Added four commands to integrate web server with sg CLI tool:

• sg run web-standalone
• sg run enterprise-web-standalone
• sg run web-standalone-prod
• sg run enterprise-web-standalone-prod

2. Removed .env.example and dotenv package because env variables are now defined in sg.config.yaml.
3. Added global alert about API proxy usage.