Process

A new feature enabled with Company-level editing permissions

Before we could introduce the smart suggestion feature, we first needed to figure out how this new feature would be enabled. We didn't want it to be on by default for fear of disrupting the normal workflow for our very active daily users. So we had to introduce it in a way that would be highly visible, yet easy to set up. After notifying our clients about the new feature via email and in-app announcement, we placed a simple checkbox within the "Integrations" section of the app. We decided to create a new "Advanced Features" tab which would give us a location to place additional new feature enablements as well. We also added an information callout that provided a link back to the knowledge base article that was also linked via the feature introduction email and announcement.

In order to activate this new feature users also needed to agree to legal terms and conditions. We opted to place this information within a pop up upon selecting the checkbox for the new feature, this way it was very clear for users what they needed to do first before opting in to this new feature.

Then upon returning to the page, we now showed the feature enabled, who it was enabled by, and when. Additionally, we also showed that this feature was enabled within the "Product Setup" pages, both as an additional verification point, and as a way for others to see this who did not also have permissions to enable "Integrations". The feature would also show as enabled within the "Integrations" tab while setting up a new product for testing. Another location where this was visible to all, regardless of roles or permissions. Here, users who had permissions to set up new products could choose to enable or disable this feature per product, even if they did not have permission to enable the use of smart suggestions within the "Integrations" page. Finally, in order to activate the new smart suggestion feature, we placed a button labeled "Smart Suggestion beta" in the top right within the header where we placed all of our prominent, most important page related actions. Adding the phrase "beta" to this made it clear to the users what to expect and that results may vary depending on how ChatGPT processed their text.

Showing all Step Smart Suggestions at once was key

After multiple design reviews with our users it became clear that they heavily favored being able to see all options at once rather than just the text box they were focused on. The approach I took to this design was to take our existing page and find a way to split it horizontally without breaking the page. To do this I worked within the constraints of the recent header design update and split the page in half just below that so that as the user scrolled down the page, a follow-up update would leave the header fixed so that context would not be lost as to where you were.

Robust Yet Simple

At the top of the page I reiterated that the view they were seeing is that of our new "Smart Suggestion" and again linked back to our knowledge base article. Then I made it very clear which side of the page represented which half as I added the labels "original text" and "smart suggestion text" and also made these labels fixed and pinned to the top of the page as the user scrolls down. Making each option a radio box meant it was very easy to select between the two options available. Additionally, we allowed the text boxes to remain editable in case a user wanted to make edits to either side.

The left side, original text options were selected by default for ease of completion. I used our light blue background with a darker blue border to show as the hover style when the user highlights over the selection. Once a selection is made, the box now appears as a slightly darker blue background, and removed its border making it clear it was the active choice. Finally, in order to speed up the selection process I added a button to the fixed label section that allowed the user to "Accept All" of the smart suggestions being offered on the right side. We decided to apply smart suggestions not only to the test case steps, but to the settings page as well, suggesting better test case names and descriptions; which were often the things testers read first to decide if they wanted to take on the testing for that request.

Alternative Designs Found To Be Less Effective

During the wireframing brainstorming phase I had developed a couple of other designs with similar effects. One approach had all of the content suggestions aggregated but this meant that the left and right sides often did not match up as there would be content like images and similar on the left, but no suggestions for images so the text would appear much shorter and we found that it was much harder for users to cognitively process. Another design approach we had was to only offer up suggestions as a user went through and selected each section on the left. This however felt much more tedious and wasn't easy to see the big picture where sometimes a smart suggestion would break a set of instructions into multiple steps. Ultimately we landed on this solution where users could see all smart suggestions offered up at once, while keeping them aligned vertically with their left side counterparts they were meant to replace.

Asking For Timely, Relevant Feedback

After users completed their selection process for both the steps and settings tabs, they could save their selections. Once saved, a modal would pop up asking the users about their experience. We wanted to keep this timely and relevant, but we also didn't want the user to feel obligated to do so. Presenting it this way increased the feedback we received on the feature, but still allowed users to opt out of leaving feedback if they wanted to keep focus and move to creating the next test case using smart suggestions.