If you’re reading this it might be because you’ve written something like this in your answer and tried to fob it off as critical thinking:
“One of the limitations of this study was that it has a small sample size and so lacks population validity.”
The thing to remember when trying to show critical thinking is that it takes time. It’s not easy. It’s what separates the students who get 4s and 5s from those who get 6s and 7s. If it was easy as writing the statement above, a lot more than 4% of IB Psychology students would get 7s. But it’s not that easy.
If you’re wondering whether or not you’ve made a good critical thinking point, ask yourself honestly: how hard did I have to think to come up with that idea?
Read more:
- So you want to assess ecological validity?
- How to evaluate any study in 3 simple steps
- So you want to assess ethical considerations?
What is “population validity?”
What we’re assessing with population validity is generalizability: can we generalize (apply) these results to people beyond the sample? In other words, would I expect that the same results would happen if the procedures were carried out on a different group of people?
For example, most psychological study participants are college-aged, middle-class, white, Americans. Would results from a study using participants like this be generalizable to all Americans? What about people from other cultures?
Population validity is a type of external validity. This means that we’re assessing the extent to which the results from the study are valid and applicable beyond the study itself. A small sample size may affect this because the smaller the sample the more potential there is for individual results and outliers to affect averages. The larger the sample the less outliers will impact the results.
If you are evaluating a study conducted on Americans and you come from a different culture, one way to start thinking about population validity is to wonder whether you’d expect the same results in your culture.
3 steps to assess population validity
Step One: Knowing the participants
As with all critical thinking, it begins with asking some questions. An important one is:
- Who were the participants in the study?
This is where it’s important that you know the methodology. What characteristics do you know about the participants. This could be their gender, nationality, age, cultural background, race, religion, health background, etc.
I would recommend focusing on the characteristics of the participants instead of the number of participants, since it gives you more potential to show deep critical thinking.
Step Two: Questioning generalizability
After you’ve identified key characteristics, the second question to ask would be:
- Who are we trying to generalize these results to?
Here’s where you have to identify a second group because population validity is about seeing if the results can apply from one group (the sample) to another population (group of people).
The second group should be related to the first to make it easier to explain. For example, if you identify that the sample had a mean age of only 24, you can start thinking about why these results might not apply to an older group of people. If they were all females, why might we question the validity of generalizing these results to males? If the participants were white Americans, why might we not expect the same results with other cultural groups?
It’s fine to ask questions in the “red zone” critical thinking areas of your essays – but it’s far better if you can answer your own questions, too!
Step Three: Explaining issues with generalizability
Now that you’ve asked a good question about a group that we might not expect the same results with, it’s time give some reasons why you might not expect the same results. This requires you to know something about the group in question and to really think carefully about why the results might not apply.
If focusing on culture, why might we not expect the same results with a different culture? Do they have different values that might have an effect?
If only one gender was studied, why might we not expect the same results with the other gender? Is there something biological that could be an issue, or perhaps the way they think might be different.
Below you’ll see a good example of how to explain an issue with generalizability.
Examples
The following examples are based on Radke et al.’s (2015) study that shows increases in testosterone can increase amygdala activity when we’re responding to a social threat. Imagine this has just been used in an essay explaining how testosterone can affect aggression. The answer is now moving into the counter-argument stage of the essay.
Weak: One limitation of this study is that the participants were all females. This means it lacks population validity as we can’t generalize to males.
Better: One limitation of this study is that it might lack population validity because the participants were all females. Could we expect the same results with males? There are a number of biological differences between males and females that could affect these results. Males may also be raised differently so they have different attitudes about aggression, so this could affect the relationship between testosterone and the amygdala.
Best: One limitation of this study is that it was conducted on all females so the results might lack population validity. One reason we might not expect the same results in males is that testosterone is the male sex hormone and more is produced in the testes in males than the ovaries in females. Because males have naturally higher levels of testosterone, perhaps their brains would respond differently as they’ve adapted to these levels over time. We know that the brain can change as a result of experience (neuroplasticity), so it’s plausible to think that maybe male brains have adapted to higher levels of testosterone than females and so the same reaction might not be observed.
There are a few things I hope you can learn from this explanation and the above examples:
- Critical thinking requires critical thinking!
- Start by asking questions
- Critical thinking is dependent on knowledge: if you don’t know the methods of studies, you can’t critically evaluate them
- It’s OK to ask questions, but it’s even better if you can answer them
Was this helpful? Hit the “like” button if it was.
Got a question or feedback? Post it in the comments.
Travis Dixon is an IB Psychology teacher, author, workshop leader, examiner and IA moderator.