An expert in evidence-based medicine, doctor and professor Luis Correa speaks on the importance of a basic knowledge of research methodology to avoid scientific fake news
We are often impacted by”bombastic” health news, sometimes even with the seal of approval of a renowned foreign university: “research claims that eggs are bad for your health”. But what is our surprise when, a few weeks later we stumble unto another headline: “eating two eggs per day guarantees the health of your heart, claim researchers.” Viver Bem has already published reports on the issue, and, definitely, the egg is yet to be considered a villain by scientists.
That happens, according to evidence-based medicine expert, doctor and professor Luis Correa, a director of the Evidence-Based Medicine Center, because we cannot identify whether the specific research is based on scientific evidence. “In order not to be fooled, always be skeptical,” teaches the doctor. Here’s the full interview:
How are health fake news structured?
These news are usually positive, such as ‘such thing causes cancer’, and never the other way around, ‘that other thing does not cause cancer’. A piece of negative information, even if true, is usually not interesting enough. And these fake news are like this because most literature works are done in a small-scale, taking into account the random phenomenon; and they are often biased (methodological errors), leading to inadequate conclusions.
And how can a layman identify the validity (or the non-validity) of a research?
That regards the scientific literacy degree of a society. It is something that should not be limited merely to scientists, society as a whole has to have the basic skills to identify facts from fantasy.
The first criterion is: be skeptical towards any news you read or hear, and be aware that many things, even when they involve a scientific work, may not be true. And there’s the problem: the human mind is not skeptical, it is credulous.
This is an inherent, biological characteristic: we are much more likely to believe than to doubt. We have to stimulate doubt. Then we have to bear in mind that the scientific method serves to filter our critical ability towards illusions.
What are these types of illusions?
We have basically two types of illusions, the first of them is randomness. Many things happen randomly – and not due to a particular reason investigated by researches. For instance: maybe it wasn’t that specific medication which improved the patient, he was already improving. But that becomes an illusion, especially within an extremely small sample: the smaller the sample, the larger the effect posed by randomness. Most literature works are small, and the scientific milieu of these small works is a factory is random-related illusions. A valid work must have ‘statistic power’, which is the ability to detect a phenomenon if it is true, if it exists. I want to prove a hypothesis, therefore my work needs to have a percentage (around 80 percent) of statistical power in order to be considered scientifically integer. However, the average among these small works published in the literature is around 20 percent.
And what is the effect of researches in the health area of the second type of illusion, bias?
Bias is when we tend to see certain things. For example, if I am in a study for a pain treatment, and someone gives me a medication. The patient will feel the placebo effect. If then you do not “blind” the study (“blinding” means making the patient not aware if he is taking the medication or the placebo), he might present an improvement which is not caused by the remedy, but by the placebo effect. And that is not a random event, since it happens systematically, it is a trend. And the one doing the investigation should also be blinded, because he may also unconsciously interpret that the patient improved after finding out that he is on the medication.
There is also the confusion bias: if I compare spiritualized people with non-spiritualized people, the mortality rate of spiritualized people from cardiovascular diseases is lower than that among non-spiritualized people. But we are not able to find out if this spiritualization is what affects this mortality rate, or if spiritualized people have habits, which we call confusion habits, that might interfere with the results. For instance, spiritualized people tend to have a less likelihood of being alcoholics, of smoking, and all this may interfere with the results. So everything must be taken into account.
And what are the levels of evidence in the studies?
Does the study have a high level of evidence or not? Everything which hasn’t a high level of evidence has a high risk of being a fake positive or fake negative study. A high-level study is a confirmatory study, one which I read and that conduct may influence me, one which I might turn into an idea to be used.
Studies which are not randomized should not even be considered?
They shouldn’t be news, much less be held as absolute truths. If a study is well-made, it should be published in a scientific journal, and then be called an exploratory study, in other words, the initial exploration of a question, which generates a hypothesis. In any way it should be used as the basis for a news piece to serve as truth for society. Once the press published a news piece, that ends up becoming the truth for the population. I call that scientific fake news.
What characterizes a scientific fake news piece in the health segment?
The different from ordinary fake news to scientific ones is that the former don’t have any evidence about them, someone simply made them up. But scientific fake news are worse, because there is a work which reached that conclusion, so they seem true. But either that work has been done poorly, didn’t have a randomized clinical study, or it was a very small-scale work. But the moment we see it published in a scientific journal, that pseudo-values that idea.
The fact that a research was done at Harvard or any other important university is not crucial to determine if we have a truth in the health area?
Precisely, of course such researches have a scientific value, which perhaps could be a path to be followed, but most of these hypotheses generated by initial works do not end up becoming a true hypothesis. They have their value as hypotheses, but not as news. Whenever you turn them into news, you generate hope, which is even cruel, because it often involves treatments or a cure for a disease. There is a sort of evilness in these scientific fake news.
Therefore, in order to avoid scientific fake news, the first thing one should do is to doubt, to be aware that the world is a factory of illusions, to identify whether the work is considering the random and bias factors, check the size of the work and the methodology applied. The journalist should be trained to identify it, and, therefore, have a stronger basis to write news to the lay public.
Regarding doctors, do you believe we are witnessing today an updated generation, able to identify new pieces of evidence, or most simply follow the guidelines and ignore scientific factors?
Even for doctors being scientific is anti-natural, because one must be skeptical. And being skeptical is not to be that one who believes, but to be that who is cautious to believe. Therefore, it is not natural, it is kind of annoying to be the one who never believes in things. If you are a doctor, if you have an office, it is only natural that you should believe in what you do. We have a culture of believing in whatever people who are our reference, our parents, teachers, have taught us. And doctors are human beings, if something is in the guideline they will follow it, because they think there’s nothing else to do. And there are many things in the medical guidelines which have no evidence, because these guidelines have been made by human beings, who are also prone to failure. Therefore, there is still an extremely innocent view from the medical community when it comes to seeing this need for skepticism. It is an evolution we need to have as a society, just as much in medicine as outside it.
READ IT IN PORTUGUESE:
Especialista em medicina baseada em evidência, o médico e professor Luis Correa fala sobre a importância do conhecimento básico em metodologia da pesquisa para evitar fake news científicas