blob: Syntax highlighting for CodeMirror blob view
Created by: fkling
This is the follow up/improved version of #39116
This commit adds support for syntax highlighting from lsif/scip data that is returned for some languages. Updating the backend to return lsif/scip data for every language is done in a separate PR (#39264), but this PR does not depend on it (if treesitter highlighting is not configured there simply won't be any highlighting).
Syntax highlighting is done by an extension that takes the JSON encoded scip data and converts it to something CodeMirror can understand. Decorations are only generated for the lines that are currently rendered.
I originally converted line/column ranges to document-offset ranges and used binary search to find the relevant ranges for the currently rendered lines. However the overhead of doing the line/column -> offset conversion was noticeable for large documents, but especially unnecessary for those because only a small subset of lines would be visible.
Then I changed to an approach that would use the data sent from the server as is (to avoid creating additional objects in memory and GC-ing the JSON decoded data) and only add a simple line index. The conversion from line/column to document offset is now delayed until the moment the decorations are created. This only works under the assumption that the server sends back the ranges order and without overlap.
Some of the changes I made (e.g. exporting the replaceValue function)
are not relevant anymore for the final version, but I'll leave them in
for completeness.
Other auxiliary changes in this commit:
- Changed the base CodeMirror hook to allow for "manually" dispatching a transaction to update the value. Without this we would trigger at least two transactions when loading a file: One for updating the document and one for updating the syntax highlighting. Now we can do both in a single transaction.
- Added a folder for CodeMirror blob extensions and moved the line numbers extension there as well.
Demo switching back to a ~15k line document without (left) and with (left) CodeMirror:
https://user-images.githubusercontent.com/179026/180868545-b48e6f0f-b00b-4992-91f5-c510c059fc0f.mp4
Demo of the optimized highlighting approach on a ~50k line document. First the previous approach with pre-processing the data upfront, second the final version. Note how there is often a slight delay between clicking the back button and rendering the document. It feels faster with the second version:
https://user-images.githubusercontent.com/179026/180868718-7873590e-e8c9-4de5-8bd0-68097866286a.mp4
Test plan
Enable tree sitter syntax highlighting locally by adding the follow to the site config:
"syntaxHighlighting": {"engine": {"default": "tree-sitter"}, "languages": {"extensions": {},"patterns": []}}Then go to a file that is supporter by tree sitter, e.g. Go.
App preview:
Check out the client app preview documentation to learn more.