Process

Unlimited Data = Unlimited Possibilities

The first task was understanding how the system worked. Qanairy uses artificial intelligence to create it’s content so that meant I had to create the framework for a lot of content that would be loaded dynamically. I had to make sure that the system scaled well for large amounts of data where the exact number would be unknown and constantly changing. This led me to make the decision to start working with a dashboard layout where the internal content could live in an adaptable, infinitely scrollable frame.

Are We Speaking Your Language?

Now that we had figured out how we were going to display the data, we needed to figure out how to talk to our users. Our main target client personas are QA Automation Engineers, QA Managers, Front End Engineers, CTOs, and CEOs. We engrossed ourselves into their world to make sure Qanairy would fit right in. Quickly we abandoned our old thinking of test paths and groups/grouping. We had to make sure these critical pieces of the application matched the real world terminology. By changing these terms to "tests" and "test suites" we were able to use terms that were consistent with jargon in the QA field.

Managing Domains

For our larger clients we imagined that they might want to run Qanairy on multiple domains. By selecting "change?" next to your current active domain in the upper right hand corner of the dashboard you will be taken to the domain manager screen. While in this screen you can add a new domain, name it, and import the site's logo. Once you have add multiple domains you can use this screen to manage an select which domain you want to work on.

Containing The Data

Each test contains a wealth of information that would be very cumbersome to review if it was all accessible at once. I knew I had to come up with a solution for keeping the content condensed while scrolling through hundreds, upon thousands, of tests. My first idea was to have all the information available upon the toggling of a collapsed/open arrow. Upon reviewing the user flows I was able to determine that this would add a lot of troublesome managing of open tests when reviewing the large set of data. Instead I came to the conclusion that we should have the tests auto-expand upon click and then return to their closed state once another test is clicked on.

User Onboarding

This is all well and good for the user once they have content, but it would be a misstep to leave the empty state blank. We wanted to make it clear to the user what they should be doing when they open the software. If a user signed in to the application for the first time the test frame would display the following text. "This area will be populated by your feature tests once you have run a 'discovery'. Why don't you begin by adding your domain in the upper right corner and then selecting 'discover'." By populating this area that is blank by default we can instead introduce recommended first steps to our user. It was very important to us that we make sure our users can easily access all parts of the software. We strived to make all of our navigation and menu options either visible or easily retrievable, as is the case with the tests' dropdown.

Creating a User's First Time Experience

We wanted to make sure that Qanairy was simple and easy to use. In order to ensure this I created a scenario map for onboarding to Qanairy.

Verifying The Tests

The core function of the Qanairy system is to generate UI feature tests. Then it is the user’s responsibility to verify their test's validity on the first run and then the system would test its results against these responses upon subsequent runs. I knew I needed to create a way for users to easily review and verify their tests in a way that was easy to use and allowed for a one-click response. I created the expanded view for tests that would allow users to see the test in it’s entirety and verify its status upon first glance without having to click any levels deeper. By keeping the response level on the surface level of the tests we were able to avoid any multiple-click actions, but in order to edit any mistakes we provided a way for them to edit each test if needed.

Update Your Status

Upon conducting a heuristic evaluation we determined that confirming the validity of the test would need to be a variable that can be changed and updated. This hit both the principles of "user control and freedom" as well as "error prevention". First, if users made a mistake upon reviewing they would need the option to change something incorrect to correct or vice versa. Second, as new code is uploaded to the qanairy system it could potentially change the status of a test. For example, it is quite possible that updating a resources link will be seen as an error in Qanairy because the link points to a new path. However, you and I know that the link merely changed, but is now correct. We needed to give the system a way to see this, and users a way to control it, in order to prevent unnecessary errors. Thus the "change status" option for tests was born.

Outcomes

This initital version of Qanairy proved to be the foundation for which the next three years were built upon. To my credit, I am very proud to say that the inital UI and user flows went largely untouched, and instead became more refined over the years as the product progressed.