Currently, users need to create data series one-by-one in the creation UI. They need to define a name, search query and choose the colour. During the research, we've discovered that users would like the system to automatically detect versions based on the capture group within the search query.
Creation UI: Adding data series manually
The customer's level of satisfaction with the current solution is low, both when it comes to setup (manual) and the outcomes (tracking only what they are aware of). It's the most common feature request that would unblock a lot of sales opportunities for big customers.
Diagram showing customer value
Main differences between the creation forms for existing and new insight types lay in defining data series - multiple query vs. one query and how regex capture groups are interpreted.
Comparison of two types of insights
Creation UI: switching between defining data series manually and automatically
For the v.1 of design, I've decided to allow users switch between two ways of creating data series - manual and from capture groups - using a segmented control button.
I've tested the prototype of Creation UI with 7 users:
Taking into consideration the results of usability testing and how insights with automatically generated data series are conceptually different from the regular insights (setup, type of query, behaviour, maintenance), we've decided to create a new insight type.
The main challenge is to explain all the differences on the selection screen and do not create a hesitation moment (or a blocker) for the users who had just begun the flow.
I came up with two types of visuals I wanted to test with users: icons and charts. During the usability testing, I wanted to compare reactions to both variants (with the same titles and descriptions to keep the results fair).
Option 1: Cards with icons
Option 2: Cards with charts
Updated creation UI: detect & track insight
After making improvements to the creation flow, I've performed 6 more hallway tests with a different group of Sourcegraph engineers (due to time constraints I couldn't test with customers this time).
Updated creation flow: choosing the insight type
KPIs & metrics
Impact on company goals
Adding this highly-requested type of insight should unblock some of the sales deals and make the expansion within existing accounts easier. We will be monitoring the overall insights impact on the bottom line as well as what role 'detect & track' insights play in the process.