Best Regards,

Peter

reply: Thank you, Peter, for your thorough whipping with the wet noodle. I'll try to respond to some of your concerns.

1. I admire what Richard Wiseman is doing with respect for Daryl Bem and his paper. He's a practicing scientific researcher with a strong interest in the paranormal. He notes that Bem has provided open access to both his data and the software used in conducting the experiments. This is how all science should work. No secrets. No missing or lost data. No need to personally inspect the lab to detect deception. Etc. This makes it easy for labs to replicate and for detecting flaws in methodology.

Wiseman has detected an important flaw in Bem's work. First, he focuses on the two studies that had the most robust findings, i.e., resulted in the largest deviation from chance expectation. These are experiments 8 and 9, 9 being a replication of 8. Those are the studies he has set up a registry for, encouraging researchers to try to replicate the best that Bem has to offer. Second, he found a problem with the method of scoring in these tests that could significantly affect the statistical outcomes. He thus advises those attempting replication to change the way these tests are scored. He also advises rechecking the scoring done in Bem's experiments.

Wiseman writes:

The potential problem is in the scoring.   The experimenters used a second piece of software to score participants’ responses.  Of course, participants may have misspelled remembered words (e.g., typing ‘CTT’ instead of ‘CAT’) or come up with words that were not on the original list (e.g., typing ‘CAR’ instead of ‘CAT’). To deal with this, the scoring software was designed to automatically go through the participant’s responses and to flag up any words that were not absolutely identical to the words that were not in the original list.  The experimenter then had to go through these ‘unknown’ words manually, and either correct the spelling or tell the software to ignore  them because they did not appear on the original list.  To prevent any possibility of unconscious bias, the experimenter should have been doing this blind to the words in the  ‘target’ and ‘control’ lists.  Unfortunately, this was not the case.

When you are dealing with very small statistical deviations from chance and relatively small numbers of participants (100 in this case), a small difference of 3 or 4 cases wrongly scored could mean the difference between getting a result that's statistically significant (the goal of those trying to prove a psi effect by these kinds of methods) and one that is likely due to chance.

I grant Peter that what Wiseman writes is much more elegant and incisive than anything I have written about Bem's work.

2. I agree my tone is negative. Regarding your response to my reference to calling the "erotic stimuli" used by Bem pornographic....another reader said that when he came to that comment he laughed so hard that he shot his morning coffee out of his nose.

Seriously, I am not a psi researcher, but I have read and reviewed two books by Dean Radin, one by Gary Schwartz, and one by Charles Tart. That should qualify me for one free pass out of the Skeptic's Hall in Hell. I've read many other articles purporting to have found evidence for some psi effect.

Yes, I call attempts to replicate Bem's recent work "ludicrous." By implication, that means I'm calling Wiseman's call for replication ludicrous, which it isn't. Wiseman's approach has already paid dividends by exposing a problem with the scoring method used in experiments 8 and 9. It may turn out that there is no statistical significance in Bem's data, and the APA journal will have to recall the paper and admit that its reviewers missed this problem. If this happens, however, don't expect all those in the media who have headlined this story as "precognition proved in the lab" to recant and tell the true story.

My comment refers to the method commonly used in psi research of seeking a statistically significant result and claiming it is evidence for some psi effect. Psi researchers like Bem and Radin call such observations "anomalous" when they aren't. There is nothing anomalous about observing people making guesses about Zener cards or names. Observing one person who consistently guesses correctly would be an anomalous event. Observing many people, some of whom make correct guesses some of the time, is not anomalous. Observing numbers of people making guesses that, taken collectively, deviate from chance expectation according to some arbitrary statistical formula is not anomalous.  What I find sidesplittingly ludicrous is doing meta-analyses on numerous psi studies, most of which are too small to be of much value, to seek some statistic one can claim, as Radin often does, has a bazillion to one odds against chance.

What wouldn't be ludicrous is that if these psi researchers, using methods similar to Bem's, consistently found deviations from chance on the order of, say, 30 or 40 percent, in the group or in special individuals. But they don't. They find deviations in the 1 or 2 percent range or less (see the PEAR experiments on psychokinesis). This is called pathological science. Langmuir described typical cases as involving such things as barely detectable causal agents observed near the threshold of sensation which are nevertheless asserted to have been detected with great accuracy. I think we can extend this idea to another batch of typical cases that involve such things as slight deviations from chance involving imperceptible perceptions expressed as a statistic. The participants are never aware of feeling any psi. The only evidence that psi has occurred is the statistic, which is always barely across the threshold of statistical improbability.

3. I do agree with the quotation from Rutherford when applied to psi research. Bem and others speak of the "anomalous transfer of information," but the only evidence that any information has been transferred at all, anomalously or otherwise, is a statistic at the threshold of statistical improbability according to an arbitrary statistical formula. Psi researchers are trying to establish extraordinary claims with evidence that, at best, can be said to be consistent with their hypotheses regarding psi phenomena. Correlations with statistical improbability do not imply causality.

4. I may be negative and dismissive because I resent having spent many wasted hours reading and studying the works of psientists. I'm appalled that anyone takes the work of Gary Schwartz seriously. It is pathetic and without any merit. I'm outraged at the way Dean Radin uses meta-analyses to produce exasperatingly ridiculous, breathtakingly inane, statistics. I won't pretend that I am not annoyed by their persistence, their assumptions, and their occasional hoodwinking of respected publications to promote their fantasies. Bem is one of the more respectable psientists and he probably deserves the kind of respect Wiseman gives him rather than the derision that I heap upon him and his kind. What's done is done, though. I stand by what I wrote.

5. If precognition is proved, I will have to rewrite a bit of doggerel I composed some 20 years ago. To wit:

No silliness intended

but the future's been suspended

until the past can catch up

to where the present has just ended.

 

I've thought about various possibilities. Here's one:

Some silliness intended,

the present's been suspended

until the future can can go back

to where the past can be rear ended.

__________

* Wagenmakers et al. contend that Bem's statistical formulas are too liberal, that his work is more exploratory than confirmatory. They write: "one-sided p-values may overstate the statistical evidence against the null hypothesis. We reanalyze Bem’s data using a default Bayesian t-test and show that the evidence for psi is weak to nonexistent." I think their background is different from mine, and so is their audience. I would be more prone to note that for all Bem knows he got the results he got because Zeus willed it. I stole that line about Zeus messing with the minds of psientists from psychologist James Alcock who, I understand, has a response to Bem's paper in preparation. (update: Professor Alcock's scathing review of Bem's work has been posted on the CSI website. update2: Bem's lame response to Alcock's critique has been posted, as has Alcock's restrained response to Bem's vociferous whining.)

** Hadlaczky's paper on the failure to replicate Bem's results is much more respectful of Bem and psi research than I have been in my comments. For example, Hadlaczky writes:

For this design to be able to show precognition, several assumptions have to be made. Firstly, that precognition exists. Secondly, that time reversed effects can be achieved using the mere exposure paradigm with highly affective stimuli. This second assumption has no base however, if it is not possible to achieve regular mere-exposure effects using highly affective stimuli. Unfortunately, as of yet there have been no studies investigating this, and therefore the validity of the second crucial assumption of the PH-design is vulnerable.

Further, the PH design is not an exact copy of a mere-exposure design. Despite the main difference that it is run backwards, there are other dissimilarities like the number of exposures, the time between exposure and selection task etc. These dissimilarities could possibly lead to the conclusion that a PH-study is not a mere-exposure study run backwards at all but simply a new design, which is only slightly similar to a mere-exposure study run backwards.

Furthermore, Hadlaczky points out that even if Bem's experiments were total failures, precognition could still exist. I might add that without a plausible mechanism for how things that haven't happened yet can cause things to happen now, it will be very difficult to design an experiment that could falsify the precognition hypothesis. Psychologists have devised tests of unconscious perception from past and present experience. This is possible because we can control the causal factors we're testing, e.g., tests that establish blindsight. But designing a test that measures unconscious perception affected by the future in such a way that failure would imply that the precognition hypothesis (or any other psi hypothesis for that matter) has been falsified may require a bit more ingenuity than shown so far.

update 23 April 2011. Bems's study has been replicated by Stuart Richie, Chris French, and Richard Wiseman. According to Ben Goldacre, they re-ran three of Bem's backwards experiments, just as Bem ran them, "and found no evidence of precognition. They submitted their negative results to the Journal of Personality and Social Psychology, which published Bem’s paper last year, and the journal rejected their paper out of hand. We never, they explained, publish studies that replicate other work." Classy. Any bets on how fast it will take the same media that trumpeted Bem's work to ignore the negative findings?

Last updated 23-Apr-2011