Our client, a Fortune 500 firm, needed some fast feedback on designs and features they were considering for a new social app in order to prioritize features into an MVP. Initial development was already underway, and the project requirements were growing and changing. The window for design or feature changes was quickly closing, and by using Validately, we were able to reach the right target segment and get quick feedback. Tweet this!
- MVP definition – Which features resonate most strongly with the target user segment?
- Design/Usability – Can users intuitively navigate to accomplish these user stories?
Structuring the Tests:
Before launching any tests, we outlined our key hypotheses related to the product concept using tools such as the Business Model Canvas and the Value Proposition Canvas. We used these high level frameworks as a baseline to define more detailed test objectives for the most important aspects of our product concept. Our test objective included the who (who we were targeting), the what (what we were testing), and the how (how to run the test). Defining the test objective and success criteria up front is very important to evaluating the results later on.
Using Validately, we setup tests for each core user story and persona to solicit both feature feedback and test usability. Because interactivity was essential to our product concept, we created clickable HTML mockups with embedded video to give the user a better sense of the experience. We then defined specific tasks for the user to complete and used the survey feature to ask questions around feature feedback (ranking 1-5) and purchasing behavior.
Once the tests were setup, we used the recruitment panel to attract users who matched our personas. Before launching more broadly to panel users, we had a couple of friends and family members complete the tests to ensure they made sense.
Interpreting the Results:
Even before we ran the tests, the process of converting the designs into clickable mockups for goal-based user stories highlighted important design gaps, which forced improvements to be addressed and saved valuable development rework time.
After launching our tests in Validately, we watched the initial screen captures and quickly found that many users were not able to complete the tasks we had outlined. This helped us identify where the designs were not intuitive and led to additional design and workflow improvements.
We also reviewed the analytics and identified that there was a distinct difference in feature interest based on the different segments we were targeting, which helped us better define our user personas. By creating a separate test for each target segment, we were able to uncover these insights more easily.
- Define the concept, tests, and success criteria up front before diving into Validately – make sure you clearly understand the learnings and outputs you are trying to get
- Keep the tests simple and focused on one objective at a time – create separate tests for use cases and target segments
- Simple mocks and wireframes work for most cases, but sometimes you may need a higher fidelity mockup to properly convey the true user experience
- Be sure to view the screen captures to gain more insight behind the quantitative results and uncover potential usability issues
- Test iteratively to make sure the results you’re getting are providing learnings you can interpret and incorporate
Ultimately, Validately was an important tool to quickly and efficiently collect the evidence we needed to provide direction to our client as well as setup our other market testing efforts, including user interviews, landing pages, etc. Collectively, these efforts enabled our client to reduce time to market by 55%. Tweet this!
Zain Yousaf (email@example.com) is a Managing Consultant at LeanCog, a product development and innovation firm that helps companies iteratively develop, test, build, and launch new products using lean methodologies.
email Sales@validately.com for details