If you’re reading this it’s probably because your teacher has sent you here after reading something in your essay along the lines of, “this experiment was a laboratory experiment and so it lacks ecological validity.”
This throwaway sentence makes teachers and examiners groan (and not award you any marks, just so you know). So I’m here to help show you how to turn this phrase into an awesome evaluative explanation that will set your essay’s critical reflections apart from others.
To begin, let’s have a bit of a recap of some key terms and concepts.
Note: this post has been edited for corrections since the original posting. Thanks to Hugh Coolican for the clarifications.
What is ecological validity?
Ecological validity refers to the extent to which the findings and conclusions from a study can be generalized to a different setting based on the environment of the original study.
Later in this post you’ll see some examples that will make this definition a little more clear and concrete. If you’re a biologist you’ll be familiar with the term “ecology,” which refers to the environment. A “valid” result or conclusion is one that is sound and has merit. So ecological validity is about assessing the validity of a study’s findings based on the environment in which the study took place. If there are reasons to suspect the environment might influence the merit of the conclusions of the study (in terms of being able to apply them to other environments), the ecological validity may be questioned.
This is probably why you’ve associated ecological validity with a laboratory experiment: laboratory experiments take place in controlled, artificial environments, so this is not a very natural environment and so whether or not the results could be generalized to a real life environment might be questionable. Your original statement, therefore, might not necessarily be “wrong.”
Edit: The term validity (and ecological validity) should be applied to the results and/or conclusions of the study – not the study itself. Instead of saying “the study may lack ecological validity,” we might say “the results may lack ecological validity.” A slight but significant difference.
But just because an experiment takes place in a laboratory, doesn’t necessarily mean its results lack ecological validity: it might depend on the environment you’re trying to apply the conclusions to.
This video explains ecological validity…
Remember that you do NOT need to make a definitive judgement on the validity of the study’s results: you should raise questions and possible issues with the methodology. Keep reading and you’ll see how to question ecological validity of results in a way that will impress examiners.
Why is ecological validity important?
Ecological validity is related to the broader concept of external validity. When evaluating quantitative studies (e.g. experiments), it’s important to think about the extent to which we could generalize the conclusions to situations, settings and people beyond the context of the original study. This is what external validity refers to: the “external” part is referring to the contexts beyond the original study. Other key concepts related to external validity are population validity (participants) and mundane reality (procedures). A good starting point for assessing external validity is to think … “could we expect these sames results if…”
Or you begin with one of my favourite phrases, “I wonder…”
Assessing ecological validity is an important skill to work on developing for a number of reasons. For one it requires practicing abstract thinking which is higher order cognition that can improve your overall thinking skills. It’s also important because after you graduate high school you’ll no doubt be required to assess the validity of conclusions from some kind of research at some point, either in your personal, professional or academic life. Being able to independently assess the validity of conclusions from research will help you beyond the IB Diploma.
Oh, and it can also help in your exams.
How to assess ecological validity?
My first piece of advice is this: ask questions before making statements.
I tell my students to think of critical thinking as more like critical reflection. If you’re evaluating a study, it means you’ve just used the results from that study to support a key point you were arguing in your essay. That point you’ve just made is essential, because it shows you understand a core concept in psychology. And so if it’s relevant to the question, you want to show you can independently and critically reflect on your own argument. In the case of evaluating a study based on the validity of the results, you’re critically reflecting on the evidence upon which you’ve based your argument (the results from the study are the evidence).
So this reflection requires you to ask some questions first. I’ll take this time to remind you that a thorough assessment of ecological validity is not easy. It takes serious thinking.
To continue the question from earlier, “could we expect the same results if…” our “if” is based on the environment when assessing ecological validity. So a good guiding question when assessing ecological validity is to wonder, “could we expect the same thing to happen if this occurred in a different environment?”
If ecological validity refers to the extent to which findings from one environment could be applied to another, some logical questions need to be asked:
a) What is the original environment of the original study?
b) To what other environment/s are we trying to apply these conclusions?
The answer to (a) should be quite straightforward because it’s described in the study. However, (b) will require you to really think carefully as there are multiple possibilities – this requirement of your own thinking is why the IB calls it “critical thinking.”
After you’ve got an answer to (a) and (b), you now need to think of reasons why the characteristics of environment (b) might be different to (a) in a way to suspect that the same results might not occur in this environment. This is where your description and explanation comes in. Good critical points still benefit from description and explanation. Remember that explanation is all about showing how things are related, so you first need to describe the relevant characteristics of the environments before you can show your critical reflection. After you’ve described the relevant characteristics, you can explain why characteristics of environment (b) might affect our expectations of getting the same results from environment (a). Or you may raise questions about environment (a) in relation to other environments.
This is all very abstract, so let’s look at some concrete examples based on three key studies:
- Loftus and Palmer’s (1974) experiments on the misinformation effect
- Bandura’s Bobo Doll experiments
- Passamonti et al’s (2011) experiment on serotonin and brain function
Example One: Loftus and Palmer’s (1974) experiments on the misinformation effect
Weak Evaluation: While this experiment shows that leading questions could act as a form of misinformation and distort the memory of a car crash, the study took place in a laboratory and so its results lack ecological validity.
Strong Evaluation: While this experiment shows that leading questions could act as a form of misinformation and distort the memory of a car crash, the study took place in a laboratory and so we might question the ecological validity of the results. The participants were viewing car crashes in a room on a television screen, which is quite different to witnessing a car crash in real life. If a car crash happened in real life and the witnesses observed the crash, this might be quite shocking or startling to witness (much more so than watching a video). The emotion involved in seeing this happen in a real life situation might be a variable that could affect the reliability of memory in this situation. Studies have shown that cortisol (a stress hormone) secretion during stressful events can facilitate the formation of emotional memories. Real life environments may involve more stress and emotion, thus higher cortisol and so the effects on memory of the leading question may be reduced. There might also be other factors accompanying the crash that might reduce the effects of the leading question, such as the sound of the impact. These sounds in real life would be quite different to those heard from the videos. Eye-witnesses might be able to recall the speed (or presence of broken glass) more accurately because the accompanying sensory details (e.g. sound of the collision) might facilitate more accurate recall.
If you study the role of cortisol on facilitating memory of emotional events you might be able to see how drawing on knowledge from other parts of the course could also help the evaluation. Even without this additional detail, this is still a much stronger assessment of ecological validity than the original.
Notice how key characteristics of environment (b) (a real life witnessing of a car crash) are described, including the level of emotion that would accompany seeing a crash. The emotional impact based on the environment is the key detail that is different between both environments and this has been described. Then this is explained as the reason why this might affect the generalizability of the results is included.
Let’s look at another example…
Example Two: Bandura’s Bobo Doll experiments
Weak Evaluation: Bandura’s experiment took place in an unnatural environment and so its results lack ecological validity. This makes his conclusions about learning to be aggressive through observation invalid.
Strong Evaluation: Bandura’s experiment took place in an unnatural environment and so perhaps we can question the ecological validity of the results. This makes his conclusions about learning to be aggressive through observation invalid. When children are watching TV at home, they have many more distractions around them and they might not practice what they’ve just observed straight away. For instance, in Bandura’s experiment they observed the model and then were directly placed in a small room with toys. This gave them a chance to immediately practice what they had just observed. At home after watching TV, there might be a number of different things that a child might decide to do (e.g. go outside and play, help Dad with dinner, etc). This would reduce the chances of being able to immediately practice and reinforce the behaviour just observed, perhaps reducing the effects of the modeled aggression. After watching violent TV, there is also not often a perfect replica of the modeled situation. By this I mean in the study they observed someone hit a unique item (inflatable clown) and then they were put in a room with the same item. If a kid watches a cartoon where an anvil is thrown on someone’s head, they probably wouldn’t have an anvil in the same room to copy this observed behaviour. Here we can see that the validity of Bandura’s results may be questioned based on the fact that the experiment was conducted in a setting that enabled kids to immediately practice what they had just observed. Whether or not we could expect these same results in other situations (e.g. a living room at home), could be questioned.
Once again, the critique is developed by identifying the characteristics of the setting upon which we’re questioning if the same results would apply. The reasons why these characteristics might influence our expectations of observing the same phenomenon that was shown in the study are explained. Definitive judgments are not made, but questionable issues are explained.
As you can see, there’s an awful lot of thinking that has to happen here. First, it’s essential that you understand the conclusions of the original study. In both these examples, if you couldn’t explain how leading questions distort memory, or how Bandura’s study shows observation of aggression could lead to imitation, it would be impossible to provide a thorough assessment of ecological validity.
You have to know and understand the study before you can evaluate it.
Both of these examples include thorough explanation. So let’s look at one that asks questions as well.
Example Three: Passamonti et al’s (2011) experiment on serotonin and brain function (read summary here)
Weak Evaluation: This experiment involved participants viewing images on fMRI screens, which means that the conclusions lack ecological validity because this is not what happens in real life.
Stronger Evaluation: This experiment involved participants viewing images on fMRI screens, which means that the conclusions might lack ecological validity because this is not what happens in real life. I wonder if we can really use these results to explain correlations between violent crime and serotonin dysfunction? Perhaps being in the enclosed space of an fMRI might have a different effect on the brain compared with being in an more open area. If serotonin is manipulated in situations where participants can walk around in wide open spaces, perhaps the results would be different. Maybe our brain might function differently in an enclosed space, as perhaps our sense of threat could be enhanced when we feel like we can’t physically move (which is what happens in an fMRI). Or what if people were in groups? The study had participants that were by themselves, in a machine. But violence often occurs between groups of people so perhaps the same results might not occur when people are in groups. So while the study can show the influence of serotonin on brain function in a controlled and enclosed environment, perhaps this might not apply to other situations.
This evaluation does not have the same depth of explanation as the other examples, but it still shows critical reflection because some very relevant and provoking questions are being asked. In my opinion, it’s equally credible as the others as it’s providing the examiner with evidence that the student can consider multiple possible variables that might be influencing the extent to which these results could apply to other situations. This is one key understanding and skill that examiners would look for if you choose to assess ecological validity.
Summary
Don’t be fooled: it is not easy to critically assess the ecological validity of findings from studies. And it’s definitely not as simple as saying, “this was a lab experiment so the results lack ecological validity.” You have to ask yourself some thought-provoking questions first and contemplate the answers to those questions.
It’s important that you comprehend the methodology of the study and that you understand the conclusions of the study before you try tackling an assessment of its validity.
After thinking carefully about many of the key studies you are revising for your exams, you might find that ecological validity is less relevant than other issues related to generalizability, such as the procedures or the characteristics of the participants. Perhaps you’d be best to keep your options open and consider multiple characteristics of the study that may influence its generalizability, not just the environment.
Evaluations of studies should be your original thinking. Examiners know what all the textbooks, websites and blogs say about strengths and limitations of studies – you’re likely to earn higher marks for being able to independently assess the validity of research and to show your original and unique assessment.
If you can give your original statements an extreme make-over and turn them into well-developed explanations, feel free to post them in the comments.
Some final reminders:
- Don’t be afraid to wonder.
- It’s OK to ask questions in essays (in the critical reflection paragraphs).
- Show your thinking.
- You do not have to make definitive judgments of studies.
- Only assess the external validity of a study if it is relevant to the question.
- You might be able to write excellent essays without referring to ecological validity (or external validity) at all – it depends on the question.
- Ecological validity refers to the environment. If you are evaluating the procedures, this is probably related to mundane reality. If evaluating the nature of the participants, this is population validity.
- If in doubt about what type of validity your critical point refers to, use the general term “generalizability” or “external validity.” Under these umbrella terms you can explain multiple possible factors that might influence the validity of the conclusions.
- Evaluation should only be tackled after you know the study, and you understand the relationship that it demonstrates (i.e. you understand its conclusions).
- Validity needs to be applied to the findings/conclusions, not the study.
- Practice.
- Practice.
- Practice.
Travis Dixon is an IB Psychology teacher, author, workshop leader, examiner and IA moderator.