Since launching in 2009, Brooklyn-based Prolific Interactive has focused on developing mobile applications for a variety of lifestyle brands. Their work has changed the way businesses like SoulCycle, Le Tote, Alex + Ani, and Lilly Pulitzer engage customers by delivering sharp e-commerce experiences.
Recently, Prolific used Validately for testing on a social platform for small business owners (SBOs) designed to connect SBOs with each other for advice-sharing, problem-solving, and networking. Senior UX Designer Nick Kroetz of Prolific sat down with Validately following the discovery phase to discuss how the research and concepting process unfolded, shaped largely by moderated remote testing.
Why did you choose to do moderated remote testing for this project?
Because the product aims to address specific problems SBOs have, we knew that qualitative feedback would be key—from concept validation through design execution. We also wanted to expand our user research base outside of New York City so we could get a better sense for the different kinds of problems SBOs experience. The truth is, small business owners are very busy people, and eliminating the cost of traveling for in-person moderated testing was a huge advantage. Validately’s moderated remote testing allowed us to collect valuable data on product and prototype experiences from demographically and geographically diverse users.
How did you source your user base for the testing?
Our timeline was pretty aggressive, and professional recruiters are able to do it much faster than we can. After we aligned on who the target user should be, based on our partner’s business goals, I did some industry research on small business owners, and crafted a screener that ensured a well-rounded participant base that allowed us to talk to the right folks. Our partner recruiting agency, Schlesinger Associates, then handled all of our participant communication and scheduling logistics, which was key for enabling our team to stay focused on the concepts.
At what points in your development did you make use of remote testing?
We started our process by pinpointing the problem that needed to be solved. We knew the app needed to be different from desktop and web experiences, and provide a unique utility that made more sense for a mobile product. And it needed to align with our partner’s overall goals for where the platform is headed. We conducted user interviews at the outset that involved more conceptual questions that targeted our primary assumptions. Doing those interviews up front helped us understand what we did and didn’t know, while also getting indirect feedback on feature ideas.
After gathering this feedback, we started iterating on the concepts until we reached a point in design execution when we could bring a few applications back to users to test the flow, features, design, and overall user experience. This is where remote testing was key; we were able to elicit meaningful feedback on the product and gauge their reactions. What we saw were both outliers and patterns, which we were then able to parse by pinpointing the features that resulted in desired reactions.
How was Validately particularly helpful in gathering this feedback?
Validately’s moderated remote testing was ideal for a number of reasons. Team members were able to tap into testing sessions remotely, and designers had the opportunity to iterate on the fly, making changes to everything from single features to entire product flow while users were testing. This made the whole process much more agile, and cut down on design review meetings.
One great example of this was our test of information sharing. A critical make-or-break aspect of the app has to do with what information—and how much information—small business owners are willing to share with each other. To get an accurate read on this, we created custom prototypes for each participant because we wanted to see how they felt about pulling in their information from existing sources like LinkedIn to accelerate the registration process. The designers were able to work in between and during sessions in another room using Validately, while populating the screens with the information they needed. We used Slack to send links back and forth while the session was happening, and it was very easy for me to drop in a new Invision link an updated prototype as the session was happening. This process allowed us to work fast enough to be able to test multiple prototypes in one day across several sessions. Other small changes were also addressed, like copy changes and button placement, increasing the accessibility of core features, and minimizing distractions caused by inconsistencies or errors that took the participant out of the illusion.
Why was this product a good candidate for this kind of testing?
The platform we were working with serves small business owners across the country, and Validately was key to understanding not just the NYC-based tech entrepreneur, but also the music business person in Nashville, the consultant in Florida, or the restaurant owner in LA. The broader perspective helped us develop a more three dimensional view of our users.
Another example of how testing feedback precipitated design updates. We found that networking was key to many of our users, but not always in the same way. In fact, we noticed two predominant categories of networkers: people who find networking difficult to prioritize, but also necessary, and others who are more open to connecting with others for both social and industry engagement. We wanted to deliver capabilities that served both groups, but wouldn’t have been able to execute an appropriate design without learning ‘why’ the feature should exist first.
What was unique to Validately’s moderated remote testing that you haven’t seen elsewhere in user research?
No other platform allows on-device, moderated mobile testing for non-native prototypes, and native apps that also captures recordings and gestures, and is able to engage other teammates using live broadcasting. Additional features like being able to make clips and taking notes with time coded comments on the on-recording notes were added bonuses.
How did you discover Validately as an option for remote testing?
In an internet search both for live broadcasting testing sessions, and remote user testing.
He has led UX on 8 different iOS and Android applications in the last few years.
Want to learn more about Validately?
Leave a Comment
Only registerd members can post a comment , Login / Register