WNM700 Module 04
Concept, Prototype, Test, Repeat
Concept
When building your ideas into interactions, there are a number of considerations. If you are designing a service, you may be writing more than drawing. If you are creating an interactive product, like a mobile app, in all likelihood you will be creating wireframes, and low fidelity paper mockups.
Evaluate Ideas
When you are evaluating ideas, you can list the criteria you may think will lead to success. There are probably a few ideas that can be dismissed out of hand due to constraints. Rather than narrow down, try and choose the idea that you think is most likely to succeed. Then, choose the idea that you find to be most innovative. You can think of this as the “Safe” choice and the “wild card”, but remember you are also selecting for maximum impact.
How to determine what to prototype
A prototype simulates that experience in some way. What needs to be simulated? Depends on the nature of the proposed solution.
Determine Goals
What needs to be determined?
If you are prototyping a ride sharing service, what needs to be determined? The use of the interface, or the willingness of a person to get into a stranger’s car? Probably the latter.
What is the interaction?
Break down the user experience into bite size chunks to determine where the greatest need is. Remember, you probably will be prototyping many different ways based on different interactions. The key is testing small pieces of the whole instead of investing heavy time and resources into bringing the entire idea to life. Build prototypes only around those details which help you address a core assumption
What needs to be quantified?
A prototype needs to be tested. And a test needs results. Which means you will need something to measure. Like duration of engagement, number of clicks, Numbers of swipe right… etc.
So, you’ll need to determine a testing methodology that results in meaningful date you can analyze and see if the solution is meeting the goals. Briefly, here are some common examples of testing methods.
Moderated vs. Unmoderated
A moderated test is one with a guide or facilitator. An unmoderated test is one in which the user is unprompted and solo.
Field Testing
A field test means you need to go to the actual location or environment that is proposed in the solution to meaningfully test some aspect of the idea.
“Wizard of Oz”
Mechanical systems need not function. A human being doing the work of the system will do.
A wizard of Oz prototype fakes functionality that you want to test with users, saving you the time and money of actually creating it. Wizard of Oz prototypes often refer to prototypes of digital systems, in which the user thinks the response is computer-driven, when in fact it’s human-controlled.
- d.School
A vs B
A vs B testing iterates on a solution by using one version as control and a second with some variable adjustment. Designers can then compare results and see which works best.
Functional Prototypes
Sometimes, to get useful data in the results, you just got to build it. Some types of interaction design thrive on functional prototyping. For example, a game must be play tested. The design objective of a great game isn’t utility, its meaningful play. It’s difficult to simulate “fun”!
Storytelling/Storyboarding
If service or experience is a big part of the proposed solution, prototyping is a bit more like casting a play. Use storytelling or storyboarding to act out the different roles.
Identify Tasks/Results
What is the user meant to do, and what is the result meant to be? List it. Now, List out the steps, and make NO assumptions.
Simulate the Experience
You may need one or many of the following cues:
Paper vs. Digital Prototypes
Modeling
Mock-up
Diagram
Role Play
Capture Data
Capture all information needed to move the design forward. Useful documentation typically involves analytics — a machine capturing interactions, like clicks, but also an Audio/Visual record is very useful.
You may also want to conduct exit interviews with test subjects, or have them give testimonials.
Analyze Results
Ok, so what did you learn? Asses and analysis of the data. Are any results surprising to you?
Seeding the next round. What new variables need to adjusted? What will you change in the nature of the test to get different results?
Parsing signal from noise. Remember, you will get garbage results, outliers, and probably a fair amount of useless data. Separating the useful bits can be a challenge.
- Read through the Class 4 Readings from DesignKit in the pick up folder. It’s a lot of material, so take your time.
- How was IDEO able to leverage prototyping in the projects Pump Away and Asili?
- Choose one of your favorite ideas from the previous research project and briefly describe how you would prototype and test it.
Activity: Hot Sauce Wizard of Oz Vending Machine
A new type of vending machine will be installed in the gallery space. It will provides hot sauce for students to try with their lunch.
Specifications:
- The vending machine will need to have some kind of control panel for making selections of the hot sauce options.
- The vending machine will need to take payment in some way
- The vending machine will need to protect the inventory
- The vending machine will need to display options in some way.
- The vending machine will need to dispense the bottles in some way.
Working in groups of 4, using the “Wizard of Oz” methodology, create a prototype that can meet the basic specifications of the vending machine.
Don’t forget!
- Determine the Goals of the prototype
- Identify Tasks and Results
- Simulate the experience
- Capture Data
- Analyze Results
Goals — e.g. choose a goal to prototype to. Some examples may be: Where should the vending machine be located? What should the dimensions be? How should the product be displayed? How long should the transaction time be?
Tasks — What should the users do, what steps should they take, and what result will they get? Write it down!
Simulate — Begin a controlled test. Remember, no prompting or moderation with Wizard of Oz! The humans act as the system.
Capture — Document audio/video with your phone. What other data could you capture.
Analyze — Once the test is complete, go through a basic analysis.
Who, What, Where?
- Where did you go?
- Who did you test your idea with?
- What were you testing for?
The Good?
- What did people value the most?
- What got them excited?
- What convinced them about the idea?
The Bad?
- What failed?
- Were there suggestions for improvement?
- What needs further investigation?
The Unexpected?
- Did anything happen that you didn’t expect?
A/B Testing
A vs B testing iterates on one design using one implementation as control and a second with a variable. Designers can then set up a test and see what works best.
- Read through the A/B Testing document from Design. Think. Make. Break. Repeat.
- What are different variables you could change on an A/B test? Give some examples from your own work.
Test This!
You will be getting practice prototyping and testing demo solutions created by a previous student.
Now that you know all about the prototyping and testing process, go down the list and see what the needs are for the project in the pick up. Read through the proof of concept, and look at the provided screens and task flows. Then, go prototype and test it! If you can’t figure out exactly how the design is supposed to work, that’s okay. Capture all of your findings in the prototype, and document the results.
Where should this product be tested?
What needs to be determined?
What needs to be simulated?
Document in your journal a section for each of the following:
- Determine the goals of the prototype
- Determine Methods of prototype
- Identify Tasks and Results
- Simulate the experience
- Capture Data
- Analyze Results
Did the proposed designs work as expected? Why or why not? What suggestions would you have for the next round?