Sampling Bias in Science: Here’s Why You Need to Go Back to the Source
Perhaps you saw the headlines this weekend: “Sexting: Just About Everyone’s Doing It,” from Discovery News, or “Online survey finds 8 in 10 adults have engaged in sexting,” from CNN. Wow – that’s remarkable! But, is it real?
After letting your imagination run wild with risqué thoughts of the thrilling new pastime, and how (apparently) nearly every is doing it, let’s pause for a minute. At this point, we should discuss sampling bias – and in particular self-selection bias.
THE PROBLEM WITH SOCIAL RESEARCH
Scientists that study populations, such as sociologists, psychologists and economists, commonly wrestle with getting a representative sample. For example, a researcher can’t force anyone to participate in a study, so some will decline – perhaps they don’t like the topic, don’t understand the topic, or are just completely uninterested. This is called self-selection bias, and when it happens, it inevitably leads to a biased or non-random sample.
This is a problem, because the results of a biased sample simply can not be used to represent the general population.
The news articles in question describe a recent sexting study. This month, two Drexel University psychologists presented preliminary results from this study at the American Psychological Association’s 123rd Annual Convention.
To learn about sexting habits, the researchers recruited participants through a new online crowdsourcing marketplace called Amazon Mechanical Turk. Through this novel platform, they developed an online project, called a Human Intelligence Task (or HIT), to find people who met certain criteria and were willing to complete a 20-minute survey on sexting. For this study, sexting was defined as “the sending or receiving of sexually suggestive or explicit content via text message, primarily using a mobile device.”
Fortunately for the researchers, 870 people agreed to participate in the study. The responders were from the US, heterosexual, 18 to 82 years old, predominantly female (58%), and white (81%). Seems like a large and reasonably representative sample, right?
Well, after the data was analyzed, a whopping 88 percent of participants reported that they had sexted, with 82 percent admitting they had done it in the last year. Furthermore, a staggering 96% endorsed the idea of sexting.
Aaah, but let’s remember sampling bias. In order to see the HIT request, the participants had to be registered as Amazon Mechanical Turk Workers and thus, were internet savvy individuals (populations who are more likely to sext to begin with). Furthermore, one can assume that anyone uncomfortable answering questions on sexuality may not complete the survey. Thus, the people who did answer the survey were very likely already open to sexting, which is precisely why they took a survey on it.
So as we mentioned before, the study population will likely be biased, and therefore, is not representative of the entire population.
But that does not mean that the research is invalid. It just means that it has limitations, which should probably be reflected in the headlines when science organizations (like Discovery News) or other media outlets (CNN)) cover the research.
And notably, in a phone interview, the lead author of the study, Emily Stasko, agreed. She stated that the findings are probably inflated by this sampling bias. Good news – I guess that means most of our grand-parents aren’t sexting after all! (But, of course, some likely are…)
THE RESEARCH HOLDS OTHER INFORMATION
In spite of these limitations, we can still learn from this group of participants. For most of them, sexting was a part of a committed relationship, but nearly half said they had sexted in casual relationships as well. While some (surprisingly) reported work or other locations for sexting, most of these activities took place in their home. And generally, the survey found that those who sext view it as a fun and lighthearted activity (not a serious pastime or significant moment of intimacy).
Regardless, the majority of respondents were in a relationship, and especially for those participants, greater sexual satisfaction was correlated with higher levels of sexting. This was true for all but the “very committed,” who demonstrated no relationship between sexting and satisfaction.
According to Stasko, others have lumped sexting with negative sexual issues, like unprotected sex and sexually transmitted infections. But, she urges, we should keep in mind that sexting can potentially have a positive side. After all, it could open a new line of sexual communication for couples.
“These findings show a robust relationship between sexting and sexual and relationship satisfaction,” reported Stasko. So,while not everyone condones the risky activity, she thinks it’s a mistake to dismiss sexting as purely problematic.
“Given the possible implications, both positive and negative, for sexual health, it is important to continue investigating the role sexting plays in current romantic and sexual relationships.”
JUST REMEMBER – ALWAYS CHECK OUT THE SOURCE
While, there are some interesting findings in the data, it’s unfortunate – but perhaps not surprising – that so many news outlets pushed the one questionable finding. Two national sources were highlighted above, but there were dozens of local news sources who made similar claims. Rarely was the potential for a biased sample discussed. Even the press release from the American Psychological Association was titled “How Common Is Sexting?” when the study simply can’t answer that question for the general population.
Statistics are everywhere. Don’t forget to take a strong look for yourself.
WATCH: A “Real World” Example of Selection Bias Is Explained
As a Futurism reader, we invite you join the Singularity Global Community, our parent company’s forum to discuss futuristic science & technology with like-minded people from all over the world. It’s free to join, sign up now!