The customer's level of satisfaction with the current solution is low, both when it comes to setup (manual) and the outcomes (tracking only what they are aware of). It's the most common feature request that would unblock a lot of sales opportunities for big customers.
Diagram showing customer value
Main differences between the creation forms for existing and new insight types lay in defining data series - multiple query vs. one query and how regex capture groups are interpreted.
I've tested the prototype of Creation UI with 7 users:
Taking into consideration the results of usability testing and how insights with automatically generated data series are conceptually different from the regular insights (setup, type of query, behaviour, maintenance), we've decided to create a new insight type.
The main challenge is to explain all the differences on the selection screen and do not create a hesitation moment (or a blocker) for the users who had just begun the flow.
I came up with two types of visuals I wanted to test with users: icons and charts. During the usability testing, I wanted to compare reactions to both variants (with the same titles and descriptions to keep the results fair).
Option 1: Cards with icons
Option 2: Cards with charts
Updated creation UI: detect & track insight
After making improvements to the creation flow, I've performed 6 more hallway tests with a different group of Sourcegraph engineers (due to time constraints I couldn't test with customers this time).
Updated creation flow: choosing the insight type