Process

Form & Test User Management: User Flow Diagrams

In order to determine what pages we needed to build out, I set about creating a user flow diagram. This path brought the user through both the Forms Management and Test Users Management pages.

Prototyping Form & Test User Management

Much of Version 4 involved establishing consistencies in padding, color, and font. I had also gone through and cleaned up the code and styles for our web app's css. This design style guide update was also timed with the release of two major back-end developments, form and test user management.

I began this assignment by creating low fidelity mockups in Balsamiq.

Form Management

When a form is detected by Qanairy the discovery halts and alerts the user that a new form has been discovered. At this point the user will edit the form by selecting the exclamation icon button and will identify the form on the next page in the form's settings.

Test User Management

Once a form has been saved and created the user is brought to the "Test User Management" screen. Here they can create any number of users each with their own username and password. This allowed companies to complete quality assurance testing on different levels of permission within their web apps.

Onboarding Gets an A/B Reskin

We first launched with the ability to automatically detect and create QA feature tests. However we soon found that some of our users also wanted to be able to create tests on their own. This led us to build a Test Recorder Chrome Plug-in that allowed users to create their own tests, right from within the browser. This integrated development environment(IDE) allowed us to offer a second solution to onboarding. In this release we A/B tested the two methods. We thought we would come out with a clear winner, but in the end we found that the response varied based on the users persona resulting in a 56% to 44% split. We continued to offer both solutions and instead saw an increase in use and activity in new users.

Add a Domain

Throughout most of Qanairy's life the domain modal remained largely unchained. Although it did feature some stylistic updates, the functionality maintained it's simplicity as a way to create a library of the user's domains.

Other prominent updates to this modal included:

• The ability to select between secure and encriptionless protocols.
• The ability to select between different browsers.
• The ability to add a company logo or image to quickly visually identify your domains.
• Added legalese to secure user liability and assure accountability.

AI Auto Test Discovery

The discovery page is where tests are loaded dynamically as they are discovered and built by Qanairy's AI. Each test must approved as passing or failing, and now users can more easily idetify the parts of their test with labels included on each element and action. In addition, pages are now represented by a screenshot of their page, and elements are screencaptured and cropped to show the focus of the test. We're also able to provide the user with different style and metadata.

Test tests on the Test page

Once a test has been approved it moves to the Tests page where it can be run again to test for differences against the control test. Here it's being shown that not only can a test's name be edited, but also it's status by browser. For example, if a developer changes the destination of a previously established control test, Qanairy will think the test failed, but you as the user can override the test status manually.

Learning From Our Customers

We used a system similar to Groove's Bug Reports and Feature Requests System to track problems and record interest in new features. This allowed us to learn what our customers wanted and begin to grow proactively as a company. One such feature that generated numerous requests was the previously mentioned Chrome Test Recorder Plug-in.

Our "Smart Naming" is an example of a feature that we came up with through customer feedback. We found that we'd receive complaints of an overwhelming amount of generated tests with a lot of upfront labeling. Our tests would autopopulate the discovery page numerically, but there was no way to quickly identify them without going into each test to see its path. In order to remedy this we developed "Smart Naming" to pull the names of the page, the action, and the element to create a rough, but highly effective naming convention. "Home page link click" is one such example of an autonamed test. Shown later at the bottom of the page is version 5 that allowed us to fix some of this disassociation between the interior content of the test and its label by splitting the view so that the test names were easily scrollable on the left and the interior content remained open and visibke on the right.

Outcomes

The next and final version of Qanairy, v5, stands below. The primary focus for this redesign was to change from a horizontal to a vertical layout. Our original horizontal layout had many issues which at this point had become apparent to us. The biggest of which was the varying sizes between images and screenshots of text links. I knew we needed a more consistent way to display this information because the current format would make the test frame jump around a lot causing user frustration and confusion. Instead I opened up the layout by simplying test paths on the left hand side and creating a more tabular layout for the test details on the right.