From Abracadabra to Zombies
Cats, Apps, and Occult Stats
12 Nov 2010. Three science stories were in the news today, each demonstrating in its own special way just how wonderfully diverse scientific inquiry is. The first story is about three engineers who have solved one of the eternal mysteries: how do cats drink? The scientists are from MIT, Princeton, and Virginia Polytechnic Institute and their study was published in Science. Impressive. I didn't realize how interesting cat tongue physics could be. I also didn't know that dogs lick water by cupping their tongues, while cats turn the top of their tongues downward when drinking. To do so they must possess an intuitive understanding of gravity, momentum, and mathematics in order to lap at just the right speed to avoid coming up empty or drowning themselves. The researchers made predictions based on their observations and tested their hypotheses. For example, they predicted that lapping frequency will equal 4.6 times the weight of the cat raised to the power of minus one-sixth. They tested their hypothesis by measuring the lapping process of a lion, a leopard, a jaguar, and an ocelot. And they confirmed their hypothesis. Had they been wrong, they would have had to come up with either a brand new hypothesis or a modification of the one they started with. Results might have been so far off that they might have had to throw up their arms and admit that their original hypothesis had been falsified.
Fortunately, the cat scientists did not drift off into philosophy of science and try to solve the demarcation problem. Is science a matter of discovery or definition? Is there some set of empirical characteristics that is essential to being a science or is science whatever the dominant cultural group calls science? Or, is the word 'science' like the word 'game'? i.e., are there many things that are called science without there being any single essential feature they all share in common? For now, these engineers are content to make observations, try to observe patterns that can be expressed mathematically, form testable hypotheses, and do experiments that test their predictions. This process seems almost novel when one considers how many scientists start with a belief and then set up an experiment to confirm the belief.
The second science story is about some work done by a graduate student at Harvard University in conjunction with a psychology professor. It was also published in Science. They developed an iApp to ask people in real time what they are feeling, doing, and thinking about. They say they found that people are thinking about what is not happening almost as often as they are thinking about what is. Whoah! But wait, there's more. They found that doing so typically makes people unhappy. They also found that people were happiest when making love, but I doubt they were happy about getting a phone call while doing so. The researchers found that people enjoy exercising and engaging in conversation, but apparently are made unhappy by getting phone calls while doing either.
The beauty of this study is that it has 250,000 data points on the thoughts, feelings, and actions of 2,250 people as they went about their daily lives. All the data was collected via an iPhone app. The sample is certainly large enough, but does it represent a good cross-section of the adult population of the world? News reports don't specify how the sample was selected. (You must subscribe to Science to see the full paper.) The conclusions aren't just about iPhone users or American iPhone users, but about "people." I'd say these researchers may be drawing unjustified conclusions from their data and that Apple is very happy they did so and will probably be willing to fund their next scientific project....as long as it involves an iPhone app and gets mentioned in all press releases. Look for further work on real time interruptus.
I've saved the best for last. Psi researcher Daryl Bem (he of Bem and Honorton fame) has had a paper on precognition accepted by a peer-reviewed publication of the American Psychological Association: the Journal of Personality and Social Psychology. He's titled his paper "Feeling the Future: Experimental evidence for anomalous retroactive influences on cognition and affect." Anomalous retroactive influence is psiency talk for precognition. (Psiency is snarky talk for psi jargon.) He modified a standard test of priming. Instead of showing subjects a word like ugly or beautiful before they viewed a picture of something like a lovely sunset or a couple making love and then testing how long it takes to respond either favorably or unfavorably to the picture, Bem showed the picture first, measured response time, and then showed the "priming" word. (You must give the guy credit for coming up with this concept.) His method is to do just the right number of tests that will provide him with a large enough sample to ward off criticism that his sample was too small and that also provides data that deviate from chance expectation with "statistical significance." His hypothesis will be that if he can show that his subjects as a group showed response times predicted by priming experiments that statistically are unlikely to be due to chance, then he has shown that precognition is occurring. Other experiments involved things like testing "recall," except the students were asked to "recall" words before rehearsing them, or asking for preference between an emotively neutral picture and its mirror image before the target is selected by the computer. (Describing each experiment would be tedious. The reader is directed here.) Of course, Bem doesn't put it quite like I have. Here's what he writes on his website:
The term psi denotes anomalous processes of information or energy transfer that are currently unexplained in terms of known physical or biological mechanisms. [RTC comments: This is not true. The term psi denotes not a process of information or energy transfer, but a statistic. Psi researchers assume that this statistic represents a real transfer of information. This is the psi assumption that Bem and many others in the field make. So, those headlines that read "Study finds evidence of precognition" are misleading. The study found a statistic that the researcher assumes is due to precognition. But who would read a press release or a story with that headline?]
Two variants of psi are precognition (conscious cognitive awareness) and premonition (affective apprehension) of a future event that could not otherwise be anticipated through any known inferential process. Precognition and premonition are themselves special cases of a more general phenomenon: the anomalous retroactive influence of some future event on an individual’s current responses, whether those responses are conscious or nonconscious, cognitive or affective.
This article reports 9 experiments, involving more than 1,000 participants, that test for retroactive influence by “time-reversing” well-established psychological effects so that the individual’s responses are obtained before the putatively causal stimulus events occur.
Data are presented for 4 time-reversed effects: precognitive approach to erotic stimuli and precognitive avoidance of negative stimuli; retroactive priming; retroactive habituation; and retroactive facilitation of recall. [Did I forget to mention that this experiment allows you to show college kids "erotic stimuli" or pornography, dirty pictures, or smut as some might call it?]
The mean effect size (d) in psi performance across all 9 experiments was .21, and all but one of them yielded statistically significant results. The individual-difference variable of stimulus seeking, a component of extraversion, was significantly correlated with psi performance in 5 of the experiments, with participants who scored above the midpoint on a scale of stimulus seeking achieving a mean effect size of .42.
Skepticism about psi, issues of replication, and theories of psi are also discussed.
There remains no plausible mechanism by which precognition (or any other psychic phenomenon, for that matter) could work. Some researchers have hung their speculative hopes on baffling applications of quantum mechanics. Some still find anecdotes the most powerful evidence. Most, however, are banking on occult statistics to baffle peer reviewers, the media, and the general public.
Skepticism is minimal in the field of psi research, except for skepticism about consensus science in physics and biology. In his discussion of skepticism in a posted draft of his paper, Bem writes:
The major theoretical challenge for psi researchers is to provide an explanatory theory for the alleged phenomena that is compatible with physical and biological principles.
You can see the psi assumption at work in this assessment. The real challenge, as I see it, is to prove that these statistical deviations from chance are not due to statistical flukes; faulty equipment; fine equipment affected by temperature, humidity, altitude, electro-magnetic interference from nearby equipment or personal items carried by subjects or researchers, etc.; errors in data recording, collection, collating, and in calculations from the data. The only alternative hypotheses Bem considers are precognition, clairvoyance, psychokinesis, and an artifact of the random number generator (RNG). The tests are administered by computers that use an RNG to randomize items in the tests. Bem posits that it's possible that participants access "already-determined information in real time, information that is stored in the computer." Or, they might be "actually influencing the RNG’s placements of the targets." Finally, he considers that the output from the RNG might be "inadequately randomized, containing patterns that fortuitously match participants’ response biases. This produces a spurious correlation between the participant’s guesses and the computer’s placements of the target picture." Satisfied that he's adequately handled these other possibilities, Bem moves on.
There are also the problems of cheating and sloppiness. Zealous psi researchers, depending on very small deviations from chance (Bem's subjects scored 1.7 to 3 percent above chance overall) to get the statistic they want, can't be assumed to always be honest and careful in the running of their labs. History is against them. Still, they do provide some side-splitters along the way. Consider: Bem concluded from his experiments "that there's a slight but statistically significant precognition effect, particularly if the person making the prediction is a stimulus-seeking extrovert." (CosmicLog on MSNBC. For a skull-splitting hoot read the commentary from RevLucifer about precognition being explained by antimatter traveling backward in time.)
We replicated the procedure of Experiment 8 from Bem (2010), which had originally demonstrated retroactive facilitation of recall. We failed to replicate the result. The paper includes a description of our procedure and analysis as well as a brief discussion for some reasons why we obtained a different result than in the original paper.
Bem considers some of his experiments to be replications of others. Replication is important in scientific experiments, but the concept becomes ludicrous when applied to the kind of statistic-seeking research Bem and other psi researchers (e.g., Dean Radin) do. We already know that if you do enough experiments there is a high probability that you will find some that yield the statistic you seek. But nobody in his right mind will identify a statistically significant statistic with real evidence of precognition. To Bem, Radin, and other psi researchers, I say: Just find one person who can reliably pick the winner at the race track or the state lottery, or give us fair warning of the next terrorist attack, and all skeptics will bow at your feet.
Members of the American Psychological Association should be embarrassed that Bem's work is being published in one of their journals without requiring him to tone down his claims. He has not shown proof of any psychic power (as he and some commentators are claiming). He's found a slight deviation from chance in his data that is statistically significant. All that means is that the slight difference he found (somewhere between 1.7 and 3%) is not likely due to chance. He's assuming it is due to "anomalous retroactive influence." The reviewers should have required Bem to admit that he does not know why his data are what they are. At best, he is justified in claiming that one possibility is that some subjects some of the time were affected by something from the future. Given what we know about nature, that possibility is not in the least bit plausible. It is disheartening to see this study published in its current form in a respected journal and to read in a blog on Psychology Today that this is "cutting-edge science." [/update]
* AmeriCares *