Ethical research in usability testing

The term "ethical research" seems pretty straightforward, so why is it such a contentious topic of conversation among researchers? Just apply what your mother taught you to your research design, right? Well, not exactly.

It's a slippery slope

Ethics are cultural, not biologically rooted. That means there is no universal definition of ethical research. Different industries, organizations, objectives, and situations have varying interpretations of what constitutes ethical research.

However, they all generally fall into a few definable and actionable practices, rather than subjective moralities like "integrity." (I don't find such moralities helpful for framing research methodology.) Morals may help you conduct yourself honorably in society, but ethics ensure honorable and accurate data collection. Ethics are more than the philosophy class you used to fall asleep in.

The five commandments of usability testing

1. Obtain informed consent

Researchers need to let participants know that we are observing them and are there to gather information. If the research team wishes to record a usability session, participants must be asked for consent. Informed consent does not, however, mean that we must share how the data will be used or even the purpose of the research. (Although, if a participant is a bit hesitant, it might be a good idea to share that information.)

The situation

You conduct a usability session with a participant and record their consent. You keep your word that the entire conversation is confidential and will only be shared with the client. But the client slips up, and the recording is accidentally shared with a third party.

Was it ethical?

Even though you did not share the data, you are at fault for not asking for permission to record the session. Recording a session makes it shareable, and you need consent to be able to share with third parties not directly involved in the research. This is not only an ethical violation — it's a move that puts organizations at risk, especially if the topic of conversation is a sensitive one, like testing a bank's online account portal. Usability testing generally involves screen-sharing, in which a researcher has potential insight into participants' previous web searches, account balances, or other things that can be inadvertently shared. Participants need to understand how they are being observed.

2. Do not proceed with malfeasance

The participant should never be in any form of danger throughout the research process. Here, "danger" is defined as any situation or event that could cause potential harm that would not have been present outside of the research activity. Even if the participant is at fault, researchers cannot enable or support a dangerous decision or action.

The situation

You're waiting for a participant to join a mobile app testing session. When they do, you notice they're in a car and driving. Instead of calling off the session, you continue along because the participant has informed you that they're only driving to the other side of a parking lot to park in the shade. You are aware that this is a bad idea, but you're desperate for the session — they're the ideal research subject, and you've exhausted your sample.

Was it ethical?

This situation may not be as blatant as asking a participant to navigate the dark web, but it is a danger nonetheless. Yes, the participant is at fault for driving while looking at their phone. Still, you enabled poor decision-making, and a potential collision leaves you at fault and your research organization at risk.

3. Respect participant confidentiality

Research participants need to be assured that their information and responses will not be shared with any entity not directly related to the research. This should be emphasized during recruiting and moderating.

The situation

You just conducted a usability session testing a website for a large retailer's digital team. You have a friend who works at the retailer as an accountant, and you discuss the research participant casually, outside of work.

Was it ethical?

While this may seem innocuous, particularly because your friend works for the client, it is unethical. You are privately relaying research-related information to an individual not involved with or affected by the research.

4. Avoid Deception

This is where there is a little bit of wiggle room. Being blatantly honest about the research can damage the quality of the data by leading the respondent. The respondent needs to know just enough to provide unadulterated feedback. This does not mean a researcher should actively tell false truths, but a lie-by-omission can be a necessary tool in user research.

The situation

You are conducting usability research sessions about a well-known company's website. The respondent is talking about their experiences, and says, "You know how you can [insert activity here] on the website?" Even though you've thoroughly navigated the website before the session, you play dumb: "No, I'm not familiar with the website. Tell me about that."

Was it ethical?

This is a lie, but one that serves to ensure quality data and get the participant to elaborate. We are in the business of storytelling — why would a respondent tell you their story if they think you already know it? This sort of lie, as it has a data-driven purpose and, more importantly, no impact on the well-being of the respondent, is not an ethical violation.

Consider this one

You are recruiting research sessions and need to fill 10 slots. The research focuses on a mobile app prototype for a small group of target users, and you know that you will not be able to fill all time slots with the given sample. So, you recruit people under different criteria to be able to fill your quota.

Ethical?

There are two issues with this scenario. First, you're lying to the client by taking shortcuts to hit your quota. That is egregious in itself. Second, you've lied to the participant about the topic. Now they've taken time out of their day to discuss something that they are unfamiliar with and are at risk of not being compensated if the client judges their feedback to be subpar. Deliberately fooling somebody into something that wastes their time or could potentially make them look bad is unethical.

5. Make clear the right to withdraw

Respondents must know that they can back out of the research at any time, and they might not necessarily be entitled to compensation if they do. Still, they always have, and should know they have, the option to back out (unless a contract stipulates otherwise, which is unlikely in user research).

The situation

You begin going through the background and criteria of a usability research session. The participant becomes a bit hesitant when you mention that they need to enter their personal account information to run the test in a realistic setting. When they ask if this is what everyone does, you simply assure the safety of the platform by replying, "Yes, every participant has done this. It's perfectly secure." 

Was it ethical?

Although you are not explicitly telling the participant that they are unable to withdraw, which is blatantly unethical, dismissing their apprehension and not explicitly letting them know they can withdraw is also unethical. Telling a respondent or leading one to believe that they must participate can open the door for unnecessary risk.

Thou shall make no assumptions

Ethical research in academia is top-of-mind with researchers and Institutional Review Boards, particularly in longitudinal studies. Some may assume that a less personal, short-lived experience with a participant in a user research session has a smaller set of boundaries and considerations, but that is not necessarily true. Any time research concerns human subjects, these ethical tenets must be adhered to, whether that is in the field or on a computer. Participants in user research make themselves extremely vulnerable by sharing personal information. To maintain the accuracy of the data and mitigate risk for an organization, research ethics need to be scrutinized to protect both respondents and researchers.

Bellomy has experienced moderators committed to protecting both research participants and data quality. Reach out to one of our experts to get started on usability research, the right way.