Company Overview
Qanairy is a cloud based AI system designed to generate automated feature tests for your user interface.
My Role
Cofounder, CPO, CMO, UX Researcher, UX Designer, UI Designer, Front-End Developer, Illustrator, Product Manager, Graphic Designer, QA Engineer
As cofounder of Qanairy, I was there from the very beginning of it's creation. We incorporated in Fall of 2016 when I joined the project as the Chief Product Officer & Chief Design Officer. My primary role on this project was to manage the research, design, and front-end development of the product's user interface.
Platforms
Web App
Tools
Balsamiq, Sketch, Invision, Brackets, Github, Trello, Basecamp, Teamwork Projects, MixPanel, Heap
Collaborators
Back-end Developer
Context and Challenge
Problem Statement
The goal of this update was to introduce new feature developments and clean up the user interface. The largest of the updates added the ability to test forms by setting guiding parameters. We also added the ability to create "test users" in order to test access and authorization.
Before: Version 1 - Mockup/ After: Version 4 - Live
From Version 3 to Version 4
Version 3 brought about a much needed design overhaul. This release introduced the debut of Qanairy's new "blurple" color scheme. While the user interface and styling stays relatively unchanged from v3 to v4, the padding becomes tighter, creating less wasted space in the header navigation.
Version 3 Demo
Process
Form & Test User Management: User Flow Diagrams
In order to determine what pages we needed to build out, I set about creating a user flow diagram. This path brought the user through both the Forms Management and Test Users Management pages.
Prototyping Form & Test User Management
Much of Version 4 involved establishing consistencies in padding, color, and font. I had also gone through and cleaned up the code and styles for our web app's css. This design style guide update was also timed with the release of two major back-end developments, form and test user management.
I began this assignment by creating low fidelity mockups in Balsamiq.
Form Management
When a form is detected by Qanairy the discovery halts and alerts the user that a new form has been discovered. At this point the user will edit the form by selecting the exclamation icon button and will identify the form on the next page in the form's settings.
Form Management
Form Settings
Test User Management
Once a form has been saved and created the user is brought to the "Test User Management" screen. Here they can create any number of users each with their own username and password. This allowed companies to complete quality assurance testing on different levels of permission within their web apps.
Test User Management
Test User Settings
Onboarding Gets an A/B Reskin
We first launched with the ability to automatically detect and create QA feature tests. However we soon found that some of our users also wanted to be able to create tests on their own. This led us to build a Test Recorder Chrome Plug-in that allowed users to create their own tests, right from within the browser. This integrated development environment(IDE) allowed us to offer a second solution to onboarding. In this release we A/B tested the two methods. We thought we would come out with a clear winner, but in the end we found that the response varied based on the users persona resulting in a 56% to 44% split. We continued to offer both solutions and instead saw an increase in use and activity in new users.
Onboarding for the first time.
Add a Domain
Throughout most of Qanairy's life the domain modal remained largely unchained. Although it did feature some stylistic updates, the functionality maintained it's simplicity as a way to create a library of the user's domains.
Other prominent updates to this modal included:
- The ability to select between secure and encriptionless protocols.
- The ability to select between different browsers.
- The ability to add a company logo or image to quickly visually identify your domains.
- Added legalese to secure user liability and assure accountability.
Adding a Domain/ Added
AI Auto Test Discovery
The discovery page is where tests are loaded dynamically as they are discovered and built by Qanairy's AI. Each test must approved as passing or failing, and now users can more easily idetify the parts of their test with labels included on each element and action. In addition, pages are now represented by a screenshot of their page, and elements are screencaptured and cropped to show the focus of the test. We're also able to provide the user with different style and metadata.
Discovery Page Test Expanded/ Closed
Test tests on the Test page
Once a test has been approved it moves to the Tests page where it can be run again to test for differences against the control test. Here it's being shown that not only can a test's name be edited, but also it's status by browser. For example, if a developer changes the destination of a previously established control test, Qanairy will think the test failed, but you as the user can override the test status manually.
Editing a test on the Tests page.
Learning From Our Customers
We used a system similar to Groove's Bug Reports and Feature Requests System to track problems and record interest in new features. This allowed us to learn what our customers wanted and begin to grow proactively as a company. One such feature that generated numerous requests was the previously mentioned Chrome Test Recorder Plug-in.
Our "Smart Naming" is an example of a feature that we came up with through customer feedback. We found that we'd receive complaints of an overwhelming amount of generated tests with a lot of upfront labeling. Our tests would autopopulate the discovery page numerically, but there was no way to quickly identify them without going into each test to see its path. In order to remedy this we developed "Smart Naming" to pull the names of the page, the action, and the element to create a rough, but highly effective naming convention. "Home page link click" is one such example of an autonamed test. Shown later at the bottom of the page is version 5 that allowed us to fix some of this disassociation between the interior content of the test and its label by splitting the view so that the test names were easily scrollable on the left and the interior content remained open and visibke on the right.
Outcomes
The next and final version of Qanairy, v5, stands below. The primary focus for this redesign was to change from a horizontal to a vertical layout. Our original horizontal layout had many issues which at this point had become apparent to us. The biggest of which was the varying sizes between images and screenshots of text links. I knew we needed a more consistent way to display this information because the current format would make the test frame jump around a lot causing user frustration and confusion. Instead I opened up the layout by simplying test paths on the left hand side and creating a more tabular layout for the test details on the right.