For years I tried to get my niece Shayle to go into physics, arguing she was eminently qualified to go into any sub-discipline she chose - from condensed matter physics, to quantum physics and stellar astrophysics. But she adamantly refused, admitting she found psychology more to her liking - and besides "there were fewer equations".
Hmmm....... well, as it turns out psychology does have "fewer equations" but let us agree it is because whatever principles it claims or asserts can be stated totally independently of a quantitative formalism. Most of the sophisticated principles of physics can't, or rather we prefer not to do so because a much greater generality and also reach of integration is enabled by posing them in mathematical form. Thus, many more problems can be efficiently solved, say using Newton's 2nd law of motion F = ma = m (dv/dt) where the derivative of the velocity is in parentheses and equal to the acceleration, a.
But now we learn, as from a NY Times report from 10 days ago ('Psychology's Fears Confirmed: Rechecked Studies Don't Add Up') psychology really isn't in the same league as physics. It is less empirical, more guided by anecdotes and subjectivity than objective analysis and tests. So, according to the author, Benedict Carey:
"The past several years have been bruising ones for the credibility of the social sciences"
The examples he lists, include:
- A top social psychologist was caught fabricating data, leading to more than 50 retracted papers
- A top journal published a study supporting the existence of ESP that was widely criticized
- The journal Science pulled a paper on the effect of gay canvassers on voters' behavior because of faked data concerns.
Now, add to that the results of a painstaking yearlong effort to reproduce 100 studies published in three leading psychology journals, finding more than half couldn't be validated and you have the makings of a disaster. According to author Carey (ibid.):
"The vetted studies were considered part of the core knowledge by which psychologist understand the dynamics of personality, relationships, learning and memory"
Which means Shayle likely would have been exposed to them at UMASS Amherst, as she pursues her Masters in Clinical Psychology.
Sadly, as Carey observes "this could sow doubt on the scientific underpinnings of the work."
Indeed.Especially as more than 60 didn't hold up. These included (ibid.):
- A study on free will
- A study on the effect of physical distance on emotional closeness
- A study on mate preference (e.g. "attached women were more likely to rate the attractiveness of single men more highly when they were highly fertile")
The new 'Reproducibility Project' found all were essentially bogus or over-stated in terms of their conclusions. They all highlighted the noxious drive to do "sexy science" over the real thing, to get ahead of peers in a hyper-competitive 'publish or perish' culture and maybe get some media attention too. This new analysis focused on studies published in: Psychological Science, The Journal of Personality and Social Psychology, and The Journal of Experimental Psychology: Learning and Cognition.
Doubtless some of the ambiguity is a result of using dodgy statistical methods as opposed to actual empirical experiments, or misapplying statistics to experiments with loose or improper controls.
What has been almost as interesting as this exposure is the response from one hotshot in the psychological research community. Lisa Feldman Barrett, Professor of Psychology at Northeastern University, writing in the New York Times ('Psychology Isn't In Crisis', Sept. 2) sought to tamp down the uproar by using the tack that psychology isn't peculiar and that replication problems occur in other sciences too and indeed it's part and parcel of science.
She argues:
"Failure to replicate is not a cause for alarm; in fact, it's a normal part of how science works".
Well, maybe - and it's also a part of how scientists who push certain edges of research fail to obtain proper replication when their reach exceeds their scientific grasp. Ask David Baltimore about that and his painted mice. Or ask Pons and Fleischmann after their claim of 'cold fusion' was shot down in physics.
The beauty of physics is, in fact, its robust quantitative basis which often provides a rigorous cross check on the experiments or observations.. Because it isn't merely the outcome of one or more experiments that must be considered, but also the governing quantitative and theoretical dynamic. How can a model be expressed in consistent, quantitative form that gives body to the interpretation of the experimental data? If that quantitative model is defective, then more than likely the whole experimental set up is as well. Pons and Fleischmann were unable to show us convincingly from nuclear theory, for example, how cold fusion can possibly work. Hence their experiments always aroused suspicion so it wasn't much of a surprise when their 'excess heating' results couldn't be confirmed.
Barrett tries to argue (ibid.) that "two well designed studies, A and B, can produce different results" but this would only be if two distinct theoretical models were being used in the physics. For example, in solar oblateness measurements, the Brans-Dicke theory yields different results from Einstein's general theory of relativity but this is more or less expected. (Since those oblateness measurement differences, the Brans-Dicke theory has fallen further out of favor while GR has survived.)
Barrett is also guilty of using a false analogy by comparing the replication failure in the recent psychology experiments to the paradigm shift from Newtonian mechanics to quantum mechanics. She writes:
"When physicists discovered subatomic particles didn't obey Newton's laws of motion, they didn't cry out that Newton's law 'failed to replicate'. Instead they realized that Newton's laws were valid only in certain contexts rather than being universal, thus the science of quantum mechanics was born."
Yes, but the discovery of subatomic particles not obeying Newton's laws didn't rest on one or two isolated experiments but THOUSANDS over years. It was the cumulative result of many different experiments (including with differing objectives and designs) from the Davisson-Germer experiment that found wavelengths associated with electron diffraction arising from a heated nickel lattice, to the photo-electric effect described by Einstein, to the more recent Aspect experiment (1982) at the University of Paris showing the nonlocal properties of quantum particles.
Barrett also asserts "the 'crisis' may simply be a result of a misunderstanding of what science is"
But I dispute that. Most professional scientists already know what science is, or should. And even if the media knotheads don't, the science community should be able to effectively correct them. What is at work here is the validity and integrity of empirical science and the public's confidence in it. This must mean that replication or confirmation of especially controversial results is at the core of what we do - not something peripheral. If I then claim that a twisted coronal flux loop will display a certain photospheric signature, say fluctuating umbral dots and an increasing magnetic field gradient (grad B) hours before an optical class 4B solar flare, then my colleagues ought to be able to make observations that confirm that. If they observe failure after failure of such flares to manifest, given my proposed observational criteria, then I cannot be allowed to get away with that.
True, as Barrett says, "science is not just a body of facts that emerge", but at the same time it is a gradual, incremental process of checking and double checking to assure that the hypotheses which we DO claim to have attained factual or full theory status, do indeed meet that exalted standard.
Otherwise, how can John or Jane Public believe anything we tell them, whether on global warming or evolution, or even the existence of dark energy in the cosmos.
In the case of psychology, and especially now its Diagnostic and Statistic Manual, this latest revelation will cause even more consternation - even after many have already read 'The Book of Woe- The DSM and the Unmaking of Psychology'.
Maybe Shayle will now finally change to physics!
No comments:
Post a Comment