How to Collaborate with Prototypes

horizontal-logo-colorValidately sat down with Victor Conesa, VP of Product Management at JUSTINMIND, the wireframing and prototyping tool, for an in-depth discussion on how to collaborate with prototypes, best practices prototyping for mobile, and more. Validately is excited to team up with Justinmind. Validately’s tools will be available to Justinmind users in early 2016.

Collaboration is crucial to successful protoyping. What are best practices for collaborating and getting buy-in from stakeholders?

Prototypes serve as a communication tool. A lot of time and energy is invested in the software industry to describe how an app or a website should look. This is generally done through specification documents and diagrams, but very few people read them, and even fewer understand them.

In the end, the only thing each and every software project stakeholder (i.e. executives, Business Analysts, UX designers, developers etc.) understands is the final application or something that looks very much like it.

Use a single tool to enable an efficient and collaborative prototyping process. At each step of the prototyping process, a good prototyping tool should enable collaboration and make sure that everyone in the team and the stakeholders are on the same page. The cost of rework should never be underestimated.

A Shared Prototype should be created at the beginning of the design process, and the whole team should be able to edit and make changes to it simultaneously.

During and after the creation process, it’s key that the prototyping tool makes it quick and easy to publish and share the prototype online or on remote user testing tools. The prototypes should be available to view in any browser or device so that the simulation is always available for any type of reviewers to give feedback.

The project owner should have complete control over the people who can access and review it. Managing reviews and comments, and being able to link comments to any specific feature or requirement is important too. The tool should also organize these elements well so that no details get lost.

When the reviewing phase comes to an end, the web or app prototyping building cycle can re-start, enriched with feedback and comments from all the interested parties. Published projects should also be easy to replace with updated versions.

For all of these reasons, we created Justinmind, so that all project stakeholders could clearly communicate between each other, and define together the application or website they want to build. And that’s also why we focus so much on all the collaboration and teamwork features (such as shared prototypes, online reviewing and commenting, versioning) that make Justinmind a great collaborative prototyping platform.

Collaboration and teamwork features are even more important in our Enterprise version, you can contact us to find out more about it.

What are some tips for mobile prototyping?

Test what you’re doing in a real mobile device. Testing mobile gestures and visualizing your app in the right resolution is key. You can’t just imagine it and leave it up to fate. In Justinmind we dedicated a lot of time and effort to make sure that this step is simple, so that immediately after defining a web or app prototype, it can be tested on an actual device by anyone. Putting the prototype in the hands of real users in a real environment also helps to detect conceptual errors and inconsistencies in the apps or websites.

Validately’s tools will be available within Justinmind in early 2016.  Click here for a demo of Validately.



What’s the difference between wireframing, prototyping, and hi-fi simulating?

Although there are many people who mix the terms, usually a wireframe refers to a schematic representation of a page or screen that does not contain any type of design. It serves the purpose of validating the elements that will go into a given screen, their size, the way they’ll be distributed, and so on.Wireframing generally doesn’t imply any sort of interaction.

However, a wireframe can’t convey important design components. Grey boxes and lorem ipsum content can’t communicate how colors and contrasts will influence content, the visual impact of brand design, the visual weight of graphics element and the visual path created by colors, contrasts and components. Additionally, a wireframe can also look dull and little attractive to a client when the real design is meant to be very effective and impacting.

That’s where prototyping comes in. Prototyping is often used to display interaction and it includes design too. According to my view, a prototype should be something that allows to test the experience of a user with the software that is being designed. In the same way as a car prototype is a real car you can sit in and drive, a prototype of an application should be something very similar to a real app, which you can actually use and test the user experience with. A prototype is functional: you click, you go to another page; you fill a form, you get real data and use them. And much more.

Hi-fi simulation is a complement to all this: you have a fully interactive prototype when you can simulate and test it in high-fidelity, i.e. when you can convey a high-tech representation of the design concepts, resulting in partial to complete functionality, ideally on the actual device. You don’t have to explain, or trust your client’s capabilities of imagining an app or website: what he sees is what he gets.


When is the right time for wireframing vs. prototyping?

Wireframes may be useful in the early stages of a project as long as no interaction plays an important role in the represented screens. For example, when designing web sites, usually the weight of the design is more on the content than on how the user will interact with it (although this is something that is slowly changing because websites are getting increasingly interactive). So a wireframe can be useful to set a number of screen layouts without having to involve the graphic designer, describing the general content, and the basic layout and interactions.

However, if the screens have a lot of interaction, you should move to a prototyping phase. Imagine Facebook defined only with boxes and arrows, clearly not enough. High-fidelity prototyping lets you show relations and rich interactions with a “file” that works just as if it was the real website or app, reacting and interacting with the user.

A final step is validation and testing, performed through hi-fi simulation with real users and project stakeholders.


How does Justmind help with wireframing, prototyping, and hi-fi simulation?

Consider the following scenarios:

  • You’re a designer, and the programmer didn’t understand that tiny little detail that makes a huge difference to the final product.
  • You’re a coder and need to know exactly what to code, and not change it every other day.
  • You’re a UX Designer, and need to test everything, but it takes too long and it’s very expensive to get it all tested.

That’s where Justinmind comes in and makes a difference letting everyone involved in a project see, understand, approve and start working on the real, final version.  Justinmind is a WYSIWYG drag and drop tool that can be used both to draw wireframes and to build fully interactive prototypes.

Justinmind also integrates with the most popular design tools, and includes advanced navigation and simulation with real data, in order to create prototypes so realistic-looking that a user might not be able to differentiate them from a real application or website. Data simulation comes in particularly handy: there is nothing like data visualization to give you and your client a clear understanding of how the project is going to work.

This way, you can test your future app or website before starting to develop it, and with no coding skills. It’s drag and drop, by learning some basic commands and tricks, anyone can create some advanced navigation interactions.

As far as hi-fi simulation is concerned, in Justinmind you can simulate your prototypes in any browser, choosing actual devices’ simulators, where you can simulate mobile gestures too. Most importantly, through our app, it is possible to simulate on the actual devices.


How can I test my Just in Mind prototype? With whom should I be testing my prototype?

Justinmind is a very versatile and flexible tool, and you can perform all sorts of tests with it.

The Simulate button lets you work with the prototype directly on your browser and realize moderate testing remotely or in person, in an extremely easy way.

The advanced commenting system, the simulation features and the possibility to upload your prototypes online allow you to share your designs and let other users simulate and give feedback without the owner of the prototype needing to be there in person.

Also, since the simulation basically uses HTML5, it is possible to integrate the prototypes with testing tools, like Validately. And this can simply be done with a click from our online platform.

As far as who should test the prototypes: the more the better. Ideally, prototypes should be tested with potential users but it’s also very useful to test them with developers, and in general with anyone involved in the project, in order to make sure that the plan has been correctly understood by all contributors.



The Value of User Testing – Q&A with Validately founder


In any IT project, user testing is essential to get feedback on the overall user experience from random users. The use of high-fidelity interactive wireframes and prototypes in user testing is changing the way IT projects are carried out as it is now possible to know how the app works for users before having even started the development process – drastically cutting rework.

A good prototyping tool must be integrated with a good user testing platform, that’s why at the beginning of 2016, you’ll have great news from Justinmind: full integration with Validately, one of the best rapid user testing platforms for prototypes and active features.

Steven Cohn, the founder of Validately, gives some practical advice on how to conduct proper user testing.

Why should a product team allocate its limited time to user testing?

Let me hit you with some industry stats:

  • 50% of the features in a typical product are unused by its customers. That’s right, half of what you are designing and building right now, will not be used by customers.
  • 30% of a typical development team’s efforts are classified as “re-work”. Meaning, re-coding an existing feature to better reflect customer requirements or to make it more usable.

Those are massive numbers. Real needle moving stats and it is wide spread. The truth is,

It is impossible for a designer or a product manager to sit in a room and guess how a customer wants to use a feature/product.

What are the different testing options? 

I’ll explain user testing using this grid:

user-testing-moderated-vs-unmoderatedWhen you are testing a new feature design, you want to learn 2 things:

  1. UI Usability
  2. Real Life Usability

UI Usability is easy to understand. Can someone go from page 1 to page 2 to page 3 to page 4 without getting confused?

Real Life Usability is more complicated. This test type answers whether a user will actually use the feature in their real life. As simple as it sounds, there is a lot that goes into that answer.

Remember this one thing when testing Real Life Usability:

What a customer says they will do, what a customer says they did and what a customer actually does are three different things.

Some example questions for when testing Real Life Usability are:

  • How will they use the feature in their daily life? How frequently? In which circumstances?
  • What do they view as alternatives to that feature? What do they like about that alternative? What do they hate about that alternative? What will make them switch from that alternative?

I can go on and on about the questions that help predict the success of a feature design. But the main point is this:

Until you put a prototype of a new feature in front of a customer, there is NO WAY to determine whether it meets the customer’s requirements to actually use it.

Simply gathering “requirements” in interviews is not good enough. Customers have to see something visually and try it out to tell you if they would use it.

With this basic understanding of the goals of user testing, the next part is how.

Moderated Testing

Moderated Testing means testing with a customer in real time. It can be remote or in person, but you are live with the customer asking them to try a prototype or already built feature and asking them Real Life Usability questions (noted above). You can also learn UI Usability with Moderated testing.

The benefit of Moderated testing is that you learn deep insights on how a customer will use a feature design. You can follow up and ask questions when a user mentions something and dig deep. However, this type of testing is a bigger time commitment.

This type of test is best done using a prototype because it’s easy to make changes. You’ll need to recruit testers that meet your persona & schedule a time to do a live session; run the tester through a prototype and ask them to complete specific tasks; ask Real Life Usability questions, noted above; take notes and record the session to share important aspects with stakeholders. It would be good to speak with 5 testers per persona. If you don’t have time or budget for 5, then 3 is better than zero.

Unmoderated Testing

Unmoderated testing is asynchronous. You set up a test, send it out to people and record their interactions, and often their voices, as they attempt to complete certain tasks. You are not live with them.

The major benefit is that it’s quick because it is asynchronous. You can get large volumes of responses while you are working on something else. However, you can’t ask follow up questions and dig deeper while a user is being tested. Thus you can really only learn UI Usability. Unmoderated is NOT helpful for learning Real Life Usability.

This test is best done with a clickable prototype, because you are certainly going to learn and want to make changes. You’ll need to recruit testers that meet your persona from either a panel or on your own. You give test takers a task to complete on the clickable prototype and measure how many can complete the task within a reasonable time period. Often you want the participant to share their thoughts out loud as they attempt to complete a task. You can ask qualitative questions afterward such as the SUS scoring approach.

Tasks should not be overly prescriptive and should not use specific words that are used in the UI. For example:

  • Bad task: Go to the Dashboard and “Add A New Story” to the queue. [since there are buttons labeled Dashboard and Add A New Story in the UI]
  • Good task: Add a story to your backlog

There are two approaches with unmoderated testing in terms of number of testers:

  • High volume click tests: typically 100 respondents and you just look at click patterns and qualitative responses
  • Talk-alouds: this is when the users voice their thoughts outloud. You only need 5 testers per persona to learn 85% of the usability issues.

How does Validately help with User Testing?

When we speak with Product Managers, UX Designers and User Researchers about why they don’t conduct more User Research/Testing, they usually say the same thing…I don’t have the time! When we delve into this this is, it’s really because the process around the actual test sessions takes longer than the testing sessions. Specifically the following pain points:

  • Finding representative customers (Personas) to take tests
  • Scheduling these tests
  • Running these tests without fooling around with recording software
  • Producing post-test reports in a shareable manner for stakeholders

Validately takes the pain out of user research/testing by automating all of these steps for our customers. We attack these pain points to make it really easy to talk to your Persona and test prototypes or live code. We offer simple and affordable solutions for both Moderated and Unmoderated testing.

How does prototyping help with user testing? 

Waiting to run user tests on coded features is an extremely inefficient and costly approach. Here’s the data (IBM):

      • 30% of a typical development team’s time is spent on re-work, meaning re-doing already coded features.
      • It is 200x more costly to change a coded feature than to change a prototype.

These stats highlight why it is a best practice is to run your user tests on prototypes. But there is another reason too. Once a company codes a feature, they become behold to it. But testing on prototypes, allows design teams the freedom to experiment with different approaches and learn the best approach before being locked into one design.

What are the features that make a prototyping tool a great user testing tool too? 

High-fidelity prototypes, like the ones in Justinmind, are always best for running your user tests. The interactions are realistic and so the user can get a feel for how the new feature will actually work. It’s also always best to run your tests in the user’s natural environment and on their normal device, which is another reason why Justinmind prototypes are perfect for user testing.

Justinmind and Validately will be joining forces starting from January next year. Stay tuned for further updates and in the meantime sign up for a Validately demo and download Justinmind to start improving your workflow!

Try it now for FREE - Get the justinmind editor and start prototyping

The Fundamental Flaw of Agile

train copy

“The train is leaving the station every two weeks, your feature is either on it or not.”

This is how a Senior PM at Twitter described the product/engineering culture at his company. That statement is quintessentially Agile…and produces a culture more focused on delivering new features rather than one focused on figuring out the right features to build.

Let’s start with what makes Agile the best software development methodology.

  1. Agile is designed to reduce the risk of large technology builds…and it delivers on that.
  2. Agile is designed to be more responsive to competitive pressures and customer demands…and it delivers on that.
  3. Agile is designed to improve product quality by shipping faster…and it delivers on that too.

But Agile is NOT designed to tell you what to build. Only how to build it faster and with less risk. Tweet this!

User Research and prototype testing is the way to figure out what you should build. A process nicely explained by LeanUX. And yes, LeanUX is the same as AgileUX.

But too often the culture of an Agile team doesn’t give a designer and researcher the time to do their jobs. Features must be shipped. And so most Agile companies have become feature factories. Like that famous Lucile Ball candy factory episode, if the designer/researcher falls behind schedule, they look like the fool. Never mind whether the feature has been properly researched and tested.

The false deadlines of Agile must be kept, and so we convince ourselves that our gut is enough, even though case after case proves that we makers are NOT proxies for our users. And the result? 30% of our engineering effort is wasted on re-work!

Don’t have time or budget for user research and testing? How much does 30% of your engineering team cost you? Because that’s the cost of rushing the design process.

Jack Dorsey recently made a massive change to Twitter’s company by laying off 8% of its staff, mostly focused on its product and engineering team. This move might have seemed insane to many. Why would a CEO, without any financial pressures, fire some of the most talented engineers in the world? Certainly it wasn’t due to skill…as these engineers were scooped up quickly.

Twitter copy

So why did Jack cut talented engineers? Obviously, I don’t know what’s inside Jack’s brain, but here’s my theory.

Large engineering teams build large products. But customers love simple. Fewer, more impactful, more simple feature set is what customers need and what Jack aims to deliver.

Translating that to your Agile culture it means fewer, but better designed (read: researched and tested) features are what customers love. Ask yourself whether your company’s culture has become a feature factory? Ask yourself whether your company’s culture gives its UX designers and Researchers the time to get it right or if they are bound by a false sprint deadline. In short…do a retrospective on Agile. Tweet this!

p.s. this might be my most controversial post yet. I am sure there are Agile consultants out there that are ready to rip me a new one…so let’s have at it. Let’s debate it. Tell me what you dislike about my straw man. Not in theory, in reality. Because I see it over and over.




UX Begins Before the Login and Ends after the Logout

There are two types of usability tests.

The first type of usability test is one which you are all very familiar with – UI Usability. This is simply the test of whether a customer can click from page 1 to page 2 to page 3 to complete a task without getting confused. It is a well documented process.

The second type of usability test is what I refer to as Real Life Usability. That simply means, in a user’s real life, can they do what you want them to do with a specific product or feature? Or is there a blocker? And even if they can do it, do they want to? Or do they solve this problem another way that they prefer? This is where user research meets user testing.

The answer to these questions is mostly driven by what happens before the user logs in and after they log out. Focusing on these times (before login and after logout) is the best way to differentiate your product and make your customers happy. Tweet this!

Let me give you a real life example that all of you can empathize with.

At Validately, we are investing heavily in taking the pain out of Moderated testing. When we first started talking with our customers about how they do moderated testing, User Researchers showed us screen recording tools like GoToMeeting,, Skype and Google Hangouts. All products that have been around longer than Validately and are very good at screen sharing.

But one question we asked User Researchers changed the conversation: “what do you do after the moderated session?”


Oh boy, did we tap into a pain point when we asked this question!

Ask a User Researcher about the process they go through to share results with stakeholders and they will describe a painful process of editing and synthesizing different data stored in multiple tools and files. The current process to share results with stakeholders in a meaningful way often takes longer than the actual moderated sessions. For a user testing tool, this is user research gold!


And so we have begun a series of features focused solely on taking the pain out of post test reporting. During our interviews, User Researchers would describe a video editing process that would take hours. So we created a simple, lightweight way to create clips in Validately. This simple feature saves hours editing videos and storing files.


Here is a tweet a customer posted on the day we released the clips feature.



While discussing what happens after logout, we heard that User Researchers often waste time trying to find specific spots in a video recording. So we added a very easy “Flag” button, which marks a specific spot during a session. This basic feature saves real time for Researchers. Here is what it looks like in the recording playback.



During our interviews about what happens after log-out, User Researchers often showed us an insane note taking and editing process. It often looked like this:


Translating this into a presentation for stakeholders often takes a lot of time. We are attacking this pain point next.

As we dig into the logged off experience, we learned that the pain points center around: finding spots in the videos that link to the notes and quick search for specific spots. To solve this pain point, we are adding a feature to enable note taking during a moderated session by the Moderator or Observers. The notes will automatically be time stamped and generate a shareable links queued to the spot in the recording where the note started. We will add hashtags for quick search too. All notes & hashtags could be consumed side by side in the video (which is easily shared via a URL) or downloaded into a CSV.

Here is a prototype of the moderators view:


And here is a prototype of the playback with notes:

These are just a few tactical features, but our vision on removing the pain of generating a post-session report is much bigger. And the reaction that we get when we share these features with customers and prospects is extremely positive. It completely changes the conversation and answers how we differentiate from the many screen sharing tools in the market.

And it all started by asking customers, “What you do after you log out of Validately?”



New Features for Moderated Testing

We’re continuing to enhance our moderated testing technology to make it easier to set up sessions, get deep insights, and easily share key findings.

INTRODUCING FLAGS: Easily add a flag during during your session to mark a noteworthy moment in the test. Easily jump to a flag as you watch the playback. Above: Adding a flag during a moderated session Above: Selecting a flag during moderated session playback

Adding a flag during a moderated session


Viewing a flag during a moderated session playback

FlagsMagnified_v2-01 (1)


CONDUCT SESSIONS WITH PARTICIPANTS OUTSIDE OF U.S.: We now provide international dial-in numbers for moderated. If you don’t see a country you need listed, email

EASILY INVITE SESSION PARTICIPANTS: You can easily copy moderated session URLs and phone numbers and even type in additional info to send to participants.