Throughout my internship, I’ve discovered several notable differences in user research between how I’ve done things on my school projects versus here at WillowTree. The purpose and end-goals of the methods we use are essentially the same, but the way I have gone about them in school is very different from how professionals go about them. Here are a few of the differences I have noticed that you can expect as you start looking for research jobs in the “real world.”
User research, of course, means talking to users. But a huge part that goes unnoticed in the research process is the actual recruitment of users for interviews and usability tests. You have to find people that have experience or knowledge in what you’re researching—for example, people over the age of 60 who use a mobile banking app. That sometimes can be challenging, especially if the user you need to talk to comes from a small, limited group. You also need to find people who are willing to give their time to help you—it might range from ten minutes to an hour.
For projects at school, I use the term “recruitment” very loosely. My school, Purdue University, is located in a small college town, so it primarily only consists of college students and faculty. That being the case, it’s difficult to branch out of the Purdue circle to find users to talk to.
In the past, my “recruitment” has been asking my college friends to take up 10-15 minutes of their time for an interview or usability test. Occasionally, I have reached out via Facebook to other students I don’t know, or approached random students I find in dorm lounges or Starbucks to do some guerilla usability testing (usability testing in “the wild”). But all of this still means that I am only recruiting a very small subset of the population—college-educated millennials. Granted, it’s hard to do professional recruitment when you are a poor college student who has no money to issue incentives or use third-party vendors.
Something that I have not personally done, but my classmates have, is offer a free Insomnia cookie as an incentive to anyone who would do an interview. This method is a cheap way to help you recruit a larger number of users that you may not get without the incentive.
At WillowTree, the recruitment process is much more formal. WillowTree maintains a panel of over a thousand people to draw from for surveys and interviews, with a wider age range and overall more diversity. For every project, we create proto-personas which consist of different types of users of a certain product. We draw on our panel to find users that match these proto-personas.
We also use other methods, such as recruiting people by placing a sandwich board outside the office, and using third-party vendors to help us find users who meet the study’s qualifications and help provide geographic diversity.
To recruit users, we send out a short screener survey to make sure they qualify and fit one of our proto-personas. We also always offer some sort of incentive for those who participate, usually a $25-$50 Amazon gift card. This process helps us ensure that the users we interview will be helpful, reliable, and accurate representations of the users of the product.
Personas are used widely across the UX world. They help researchers and designers better understand who their users are, and what the users’ goals and frustrations are. This helps researchers and designers develop greater empathy towards their users, leading to better human-centered designs. There are endless possibilities for how personas are developed, formatted, and used.
In my UX process at school, I generally always develop personas from scratch. After completing interviews with about 5-10 people, my team will do an affinity diagramming session, where we write down all of the findings from the interviews onto individual sticky notes and try to group the similarities into categories. Then we’ll develop 2-3 personas from these categories.
Each persona has a name, a face, demographic information, behaviors, goals, frustrations, and a scenario describing where they might encounter problems in their day-to-day lives. We use these personas to discover friction points and base our ideas and designs around them.
WillowTree does the opposite of what I’ve done in school; instead, we first create high-level proto-personas of who we think the users will be, based on market research and background information provided by the client. We then use these proto-personas to recruit users for interviews or usability tests, and iterate on the personas as we interview real users.
The proto-personas generally include the same information as the personas I have made in school, with a name, demographics, behaviors, needs/desires, frustrations, etc. Gathering this data requires several screener questions when recruiting, so we are able to target and talk to the correct group of users.
Tools and Resources
There are countless tools and resources out there for user researchers, ranging from surveying tools to usability testing tools. While these tools aren’t required for the research process, they can help make the job for the researcher a little bit easier and quicker (which is almost necessary during sprints).
Unfortunately, most tools and resources out there are not free, so I had limited exposure to them in school. The one that I did use a few times was a surveying tool called Qualtrics that our university offers for all students and faculty. It was very helpful in gaining a wide range of general responses in a short amount of time, and also gave numerous visualizations for each question that made it easier to analyze the data we received.
For gathering secondary research (marketing and psychology research) on the same topic, Google Scholar became my best friend.
Other than those two resources, all of my research was done without the help of any tools and resources. Having done research both with and without tools, I’ve realized that research can still be just as thorough without any tools, but you’ll need to allocate more time to the project to receive strong results. Tools and resources are there to enhance the process, not replace the researcher.
On that note, I’ve really loved having the opportunity at WillowTree to explore new research tools. I’ve used two different surveying tools, SurveyMonkey and Typeform, for screener surveys and getting data for the Kano model. These tools give us from hundreds to thousands of results in a matter of a few hours.
Another tool that researchers often use is a usability testing tool. Sometimes it is not the best course of action to have users come to the office for an in-person usability test or interview, perhaps due to time constraints or wanting to get feedback from users in different cities or countries; in those cases, we need to conduct remote usability testing.
The testing tool I’ve been exposed to at WillowTree is Validately. Validately acts as both a recruitment and remote usability testing tool, in that we use it to recruit people who fit our proto-personas, and then they test the product remotely. This can be extremely helpful when we need to test types of users that we can’t find locally, such as users who live in Europe.
Tests can be either moderated or unmoderated, meaning either a researcher is video chatting with the user and asking the user to answer questions and complete tasks live, or there is no researcher present during the test, and the user is given predetermined questions and tasks that the researcher has written up. Validately records both the user’s screen and their face for the researcher to go back and watch. This tool makes the recruiting and testing process much quicker, which is sometimes necessary when a deadline is approaching.
One obvious difference between my process at school and the process at work is the overall professionalism with how everything is done. Obviously, things are going to be more professional and thorough at work, compared to research on-the-go by a college student, but I’ll explain more in-depth how this difference applies to the steps in user research.
I’ve already discussed how the recruitment process is casual in my school projects, so now I’ll talk a bit about the actual research.
There are no fancy designated rooms or labs for interviews and usability tests, so they often take place in a lounge or library, or wherever we can find a quiet(ish) location to talk on campus. As much as I would like to designate an hour to interviewing, interviews often only last at most 20 minutes in respect to the person’s time, since they’re not getting an incentive.
As for recording, in school projects, I have only voice recorded (no video) interviews and usability tests—but I do recognize the importance in recording and capturing facial reactions. Also, all of my usability tests have used very low-fidelity prototypes (usually semi-neat sketches cut out into the shape of a phone or desktop screen). The user would tap the sketches as if they were tapping or clicking on the screen, and either I or my teammate would manually switch the screens depending on what they tapped. This method may not seem effective, but you actually get similar results to testing a high-fidelity prototype, and you don’t have to take as much time as you would designing something in Sketch and then creating links using InVision.
Something I have made sure of for school projects is to read a consent script to my users. It’s important that the users you’re testing with are comfortable giving you as much information as they can, and they know what it will be used for and who will be seeing the information.
At WillowTree, we have a designated Usability Lab for in-person research. There are two rooms: one room where the test or interview is conducted, and another room where teammates are watching, observing, and taking notes.
Between the rooms is a two-way mirror. In the testing room, it appears as a normal mirror, but in the observation room, you can see into the testing room. This allows others to better observe what is happening, while making the user feel more comfortable by not overloading the testing room. For a traditional interview, usually only one camera focuses on the user to record facial reactions and expressions. Usability tests employ two cameras: one records the user, and one records the testing device (such as a smartphone) to follow what the user taps or clicks on. Recording the interviews and tests allows the research team and clients rewatch them and pick up on things they missed the first time around.
You can obviously see that the Usability Lab makes the research seem much more professionally done, and it is. However, you can still get really impactful results without technology. While technology and equipment certainly sophisticate the research process, it’s about the researcher being able to empathize with the user and get honest feedback from the user.
While I’ve only highlighted differences, there are actually a lot more similarities between my research process at school and the one at WillowTree. In UX research, there will almost always be some types of persona, user interview, survey, usability test, and secondary research in the process, but there are a million different methods and approaches you can try to accomplish them.
In the end, all user researchers have the same goal no matter their approach: to synthesize the data they receive, to better understand and empathize with their users, and inform user-centered designs.