From Mad in America: A recent study published in the elite scientific journal Nature has found many problems with brain imaging studies in psychiatry. These studies, which compare brain structure or function with psychological issues, usually only have a small number of participants. As a result, they often lead to overly positive results which are incorrect.
Researchers have found that these positive results, even though they might be incorrect, easily get published in respected scientific journals. However, when other scientists try to repeat these studies, they often fail to find the same positive results. This has been termed the “replication crisis” in psychological research.
These studies, referred to as Brain-Wide Association Studies (BWAS), usually use small groups of around 25 participants. The problem is that the small sample size can cause results to look bigger than they actually are. This means that the links that are found between brain and psychological measures (anger, hyperactivity, autism, sadness etc.) look stronger than they actually are in reality.
In this recent study, led by neuroscientist Scott Marek at Washington University, researchers analyzed brain scan data from around 50,000 participants using large sets of data. They found that the correlations between brain characteristics and psychological states were much weaker than previously thought.
Correlations, which measure the relationship between two sets of data, go from 0 (no connection) to 1 (perfect connection). In this study, the average correlation between brain measures and psychological measures was only 0.01, which is a very weak link. The strongest correlation they found was 0.16, which is far from being clinically meaningful.
This small correlation means that many people share similar brain features regardless of their psychological conditions. For instance, individuals diagnosed with depression might have the same brain connectivity as those without depression. Similarly, people with ADHD might have the same brain volume as those without ADHD.
Often smaller studies will inaccurately report that people with ADHD or depression have different brains, but this larger study shows that MRI data is not that simple.
The reason for the difference between these findings and those of smaller studies is because of the sample size, which means the number of participants. Smaller studies often produce stronger correlations because of random chance, and only the most inflated results get published.
This problem isn’t new; brain imaging has been known to be unreliable in Psychiatry. MRI data is complex and often includes random fluctuations. This makes it difficult to see what is meaningful information and what is just random noise. Different research teams analyzing the same data have often reached vastly different conclusions.
In conclusion, this study highlights that many brain imaging studies in psychiatry, especially those with small sample sizes, may not be as reliable as people believe. The inflated positive results often seen in these studies can lead to misleading conclusions. Further, when researchers repeat these studies, they often find completely different results, which means we should be very careful when interpreting such findings.
This article was originally published in Mad in America and can be accessed here.
This is an AI generated version shortened and edited for the South Asian audience.
