The New York Times has an interesting story about how and why the plethora of studies about a wide variety of topics seem to create noise rather than create clarity. Here's how the Times frames the story:
"Dozens of studies are publicized every week. But those studies hardly slake people’s thirst for answers to questions about how to eat or how much to exercise. Does exercise help you maintain your memory? What kind? Walking? Intense exercise? Does eating carbohydrates make you fat? Can you prevent breast cancer by exercising when you are young? Do vegetables protect you from heart disease?
"The problem is one of signal to noise. You can’t discern the signal — a lower risk of dementia, or a longer life, or less obesity, or less cancer — because the noise, the enormous uncertainty in the measurement of such things as how much you exercise or what exactly you eat, is overwhelming. The signal is often weak, meaning if there is an effect of lifestyle it is minuscule, nothing like the link between smoking and lung cancer, for example.
"And there is no gold standard of measurement, nothing that everyone agrees on and uses to measure aspects of lifestyle."
Of course, this isn't just a consumer problem. Even beyond all the perfectly legitimate research, there is "a cacophony of poorly designed research, the tendency for different researchers studying the same effect to use different measurements and report outcomes differently, and researchers’ tendency to selectively report positive or 'interesting' results."
And so, the Times writes, the result often is nothing but confusion ... or worse, conclusions reached for wrong and insufficient reasons that result in actions that may or may not be positive.
You can read the entire story here.
"Dozens of studies are publicized every week. But those studies hardly slake people’s thirst for answers to questions about how to eat or how much to exercise. Does exercise help you maintain your memory? What kind? Walking? Intense exercise? Does eating carbohydrates make you fat? Can you prevent breast cancer by exercising when you are young? Do vegetables protect you from heart disease?
"The problem is one of signal to noise. You can’t discern the signal — a lower risk of dementia, or a longer life, or less obesity, or less cancer — because the noise, the enormous uncertainty in the measurement of such things as how much you exercise or what exactly you eat, is overwhelming. The signal is often weak, meaning if there is an effect of lifestyle it is minuscule, nothing like the link between smoking and lung cancer, for example.
"And there is no gold standard of measurement, nothing that everyone agrees on and uses to measure aspects of lifestyle."
Of course, this isn't just a consumer problem. Even beyond all the perfectly legitimate research, there is "a cacophony of poorly designed research, the tendency for different researchers studying the same effect to use different measurements and report outcomes differently, and researchers’ tendency to selectively report positive or 'interesting' results."
And so, the Times writes, the result often is nothing but confusion ... or worse, conclusions reached for wrong and insufficient reasons that result in actions that may or may not be positive.
You can read the entire story here.
- KC's View:
