Download the new eBook for quantitative research methods right here.
“This study was correlational but correlation doesn’t mean causation.” If you think this is critical thinking, think again! Let’s look at three ways to PROPERLY evaluate correlational studies.
A correlational study is when researchers measure the strength of a relationship between co-variables by calculating a correlation coefficient. In order to show critical thinking you must give specific reasons why we can’t deduce a causal relationship.
Read more:
- So you want to assess ecological validity?
- So you want to assess population validity?
- Lesson Idea: Thinking critically about correlations
#1. Does B affect A?
Can the relationship be LOGICALLY explained in either direction? If so, how? This is the best way for students to begin evaluating a correlational study.
In most correlational studies we make the immediate conclusion that variable A is affecting variable B. For example:
- Time spent using technology is correlated (negatively) with working memory capacity in children (see Cain et al.)
- Socioeconomic status is correlated with hippocampal development (see Luby et al.)
- Stress is correlated with cardiovascular disease (e.g. hypertension, aka high blood pressure) (see Suter et al.)
It’s logical to first conclude that variable A is affecting variable B. Most students then evaluate such a conclusion by saying “correlation doesn’t mean causation” and then move on to the next point.
However, we can show correlation doesn’t necessarily mean causation by showing how variable B might logically be affecting variable A. . This is explained best when there’s supporting evidence. Let’s look at three examples of this using the above studies.
- Children with low working memory capacity might be more drawn to technology and spend longer time on it instead of other activities. Why? Other non-tech activities like reading and puzzles or board games require strong working memory and if kids don’t have that they might not enjoy it as much as digital games that don’t require the same working memory capacity.
- Hippocampal development might be the reason for the lower socioeconomic status (SES). SES is measured through education levels, but if someone has a small hippocampus they might have learning troubles since it plays a role in memory, so they have reduced chance of going further in higher education. This could also affect job prospects as well, all contributing to the link between SES and hippocampal development.
- Cardiovascular disease could be affecting stress levels. It’s stressful having to deal with things like diabetes, high blood pressure, obesity and the risk of heart attacks associated with CVD. Therefore, it might be the high blood pressure that’s causing the stress, not the other way around.
Notice how the above explanations begin by stating the relationship in the opposite direction but go further to give a (hopefully) logical explanation as to why the relationship could be explained in both directions.
#2. What’s the Correlation Coefficient?
Students (and even researchers) love to overstate the significance of their findings. I am surprised at how many correlational studies are published despite having rather weak correlations. Remember that generally speaking, 0.2-0.4 is considered a weak correlation, 0.4-0.6 is moderate and above 0.6 is very strong. Therefore, anything around the “weak” margin should be pointed out. This is an easy way to show you understand the nature of the correlational findings and that conclusions need to be tentative.
From PHDComics. Original source.
Here are some examples from studies used in my course and resources:
- Karl et al.’s correlational study between hippocampus and PTSD (-0.28) and this was the strongest correlation found in the study (amygdala and PFC studies were also analyzed).*
- Cain et al.’s study on working memory capacity and technology use. Moderate but statistically significant negative correlation between working memory capacity and media usage • -0.27 for digit span tasks and -0.38 for n-back tasks.
- Hitchcock appraisals and PTSD symptoms: There was a moderate but statistically significant correlation between negative appraisals and PTSD symptom severity after six months (0.31).
- Moore et al. serotonin and antisocial behaviour. There was a significant negative correlation between serotonin levels and antisocial behaviour (Effect size -0.45.)
- Suter et al.’s correlational study on blood pressure and stress levels was a measly -0.12.!
*Technically this isn’t a correlation coefficient it’s an effect size since this was a meta-analysis. Whereas a correlation coefficient shows the strength of a relationship, an effect size shows the strength of one variable’s affect on another.
This is why remembering the specific correlational coefficient reported in a study is important. It helps you both show detailed knowledge of the study and gives you something to evaluate.
Wherever possible I try to include specific correlational coefficients in the summaries of studies in my books, flashcards, blog posts, etc.
#3: Is there a third wheel variable?
There might be a third variable (C) that affects the relationship between B and A or explains why the two are correlated. Variable C could be a moderating or mediating variable.
- Moderating variable: A moderating variable affects the strength of the relationship between A and B.
- Mediating variable: A mediating variable explains why the relationship between A and B exists in the first place.
This is difficult and requires careful critical thinking. Let’s look at the three examples stated above.
- Technology and working memory: Time spent using technology is correlated (negatively) with working memory capacity in children. Perhaps parental support is a moderating variable. Parents who spend lots of time playing with their kids (e.g. reading and playing games) could have kids with less time spent on technology and the non-tech games played with parents and kids influences working memory. Alternatively, a kid might spend more time with technology if their parents are busy and this lack of support might contribute to poor working memory.
- Poverty and brain development: Stress could be a third variable here that’s important to consider. Stress can affect hippocampal development as shown in animal experiments and it can also be increased by poverty (i.e. poverty causes stress). This can also affect parent-child relationships – poorer parents have more stress which negatively affects their interactions with their kids. In fact, this is exactly what Luby et al.’s study found out – stress and parenting were mediating variables in this relationship between SES and hippocampal development. This means they explain why poverty affects hippocampal development.
- Stress and blood pressure: Stress is correlated with cardiovascular disease (e.g. hypertension, aka high blood pressure) (see Suter et al.). Social support (e.g. having friends) could moderate this relationship. Stressors could be more sever and cause higher stress if someone doesn’t have social support and this increases blood pressure further. One reason for this is because social support is an important coping resource. If someone has social support they might feel like they have more resources to cope with a stressor, so it causes less stress and has less impact on blood pressure.
From Luby et al. 2013: “Conceptual Model Testing Multiple Mediators of the Hypothesized Association between Income-to-needs and Variation in Brain Volume.”
Lesson Idea
By far this third type of evaluation is the most difficult. It also runs the risk of some far out and hair-brained hypotheses. This is why it could be a good lesson idea when revising for exams – get students to find 2 or 3 key correlational studies they’re using and see if they can evaluate them using these three approaches. Have a teacher check their points to see how logical (or illogical) they are.
Travis Dixon is an IB Psychology teacher, author, workshop leader, examiner and IA moderator.