JetBrains: Fix highlight rendering for code preview

Created by: philipp-spiess

Closes #38222 (closed)

This fixes the line highlight offsets that we use to render the highlights in the native code preview area.

Some notes about the issue:

• Previously we were using byte length (String#length) to sum up the length for all previous rows and code pointer size for the offset in the current line. This is incompatible and caused the offsets to be invalid.
• Java does not understand Unicode code pointers well and the only API we have to set highlights in the code preview requires absolute byte offsets.
• So to fix the issue, this PR attempts to convert the offsets returned by the Sourcegraph API (code pointers) to the UTF byte offsets.

The conversation works by finding the offset in the line based on the unicode code pointer and then counting the bytes up to that offset.

A note about symbol search

Problem 1: Highlight offsets are broken

Symbol search does not return offsets from the API. Instead, we rely on the offsets encoded in the URL. Unfortunately it seems that these are not working for unicode characters and the offsets returned are larger than even the byte offsets of that row.

An example is the following search query: repo:^github\.com/0x03f3/php-emoji-reverse-shell$ 😍 type:symbol

If you inspect the link (ending in ?L3:72-3:76), it describes offsets from 72 to 76. However the byte size of this line does not exceed 72:

So this is unfortunately an issue from the backend that we can not easily work around with.

Problem 2: Emojis in symbol name crash the program.

Symbol search was also broken in a different way. Since the symbol name was part of the serialized preview payload but can contain emojis, it was not deserializing properly in the Java side and would cause the app to crash.

I fixed this issue by applying the same base64 encoding that we already apply to the file content to the symbol name.

Test plan

One emoji before the match r:leonardomso/33-js-concepts test

Multiple emoji before the match NestJs + TypeScript + TypeORM + Redis + MySql + Vue + Element-UI

Regular results console.log

Emoji symbol result repo:^github\.com/0x03f3/php-emoji-reverse-shell$ 😍 type:symbol

Note: As explained above, the highlight offsets for these are unfortunately broken for lines that contain non ASCII chars.

App preview:

• Web
• Storybook

Check out the client app preview documentation to learn more.