Code Insights serves new jobs to be done, like commit trends and change heatmaps, with new charting options (powered by compute)

Created by: Joelkw

Problem to solve

Customers want to use Code Insights to look at patterns in their commit history (such as "what is being committed?" "are certain teams/repos adopting larger/smaller/less/more frequent commit patterns that we would expect?" or "which areas of the codebase change often? how often are certain areas I care a lot about changing?") that compute can enable, but we haven't built this functionality yet.

Measure of success

Leading measure of success: we collect pings usage on insights created, by type. Based on the success of our capture group insight type, we would expect to see 3 customer instances create this insight type in the first month it's available, and 12 customers create the insight type by 3 months later.

Lagging measure of success: this type of insight drives 3 code insight sales.

Solution summary

There is a significant amount of UX planning we still need to put into this issue (an RFC will be started in late April) to ensure we balance bringing the power of compute to the ease-of-setup that code insights has otherwise found success with.

Broadly, the solution will involve:

a new insight type
new charting options

What specific customers are we iterating on the problem and solution with?

Impact on use cases

Code Health: Change heatmaps let customers understand which areas of the code change often, which can indicate brittle code or healthy code, depending on the use of a file (probably core architecture work shouldn't change too often; documentation being updated often is good). Commit patterns can help customers identify repos or organizations that have particularly high or low velocity, and then dig into why those repos or organizations are performing better/worse than expected.

Code Reuse: Seeing the frequency of various topics or APIs in commit messages, or frequent updating of key library files, can indicate high or low code reuse, so a customer can actually measure their efforts to increase code reuse more.

Fixing vulnerabilities: Understanding which areas of the code may be stale and haven't changed in a long time can surface potential audit/review efforts that catch vulnerabilities before they go live.

Incident response: Commit behavior helps you understand how fast/slow your response was, as well as how long after an incident you had to continue responding.

Onboarding: Seeing change heatmaps can help someone understand which areas of the codebase they should focus on learning, and commit patterns (from a high level POV, not focused on individual performance) can help indicate how successful onboarding efforts are.

Delivery plan

Initial UX/Product RFCs (End of April to Mid-May)
Implementing new compute-powered insight type
Releasing and promoting to above-mentioned customers and existing insight customers via Slack/CE channels.