Ted Goertzel, Rutgers University, Camden NJ 08102
Published in The Skeptical Inquirer18
(3): 266-272, 1994. (this is the original manuscript, prior to
any editing done by the journal.)
For a Russian translation provided by Everycloudtech,
Измерение распространенности ложных воспоминаний: Новая интерпретация "Исследование похищений НЛО
For a Bosnian translation by Vlada Catalic go here.
Hopkins, Jacobs and Westrum's (1992) assertion that 3.7 million Americans are suffering from "UFO abduction syndrome" has been criticized for methodological and logical deficiencies (Stires, 1993; Klass, 1993; Hall, Rodeghier & Johnson, 1993; Dawes and Mulford, 1993). In an attempt to remedy these deficiencies, my research methods class at Rutgers University in Camden undertook to replicate Hopkins, Jacobs and Westrum's survey. Our questionnaire included the same items which they believed were indicators of UFO abduction, but added other items designed to place their items in a broader psychological context. In March and April, 1993, we conducted interviews with 697 residents of southwestern New Jersey. Our sample was partly random and partly accidental. We began with telephone interviews of randomly selected telephone subscribers, but it proved too difficult for the students to obtain an adequate sample in that way, so we relaxed the sampling plan to allow the students to include their friends and family members. This kind of accidental sampling is not appropriate for generalizing to a population at large, but it is generally viewed as adequate for testing the validity and reliability of a measure. There is no reason to believe that systematic sampling biases were introduced which would have distorted the correlations between questionnaire items.
Despite the use of accidental sampling, the frequency of "unusual personal experiences" in our sample was close to that obtained by Hopkins, Jacobs and Westrum's representative national sample. Table One gives the percentages from both samples for the five experiences which Hopkins, Jacobs and Westrum used as their measure of UFO-abduction status. By their criteria of having suffered four or five of these experiences, 3.7% of our respondents qualified as "abductees." This number is quite high, especially considering the fact that the interviews were done in the suburbs of Philadelphia, an urban region which is not known for a high frequency of UFO sightings or abduction reports.
As Hall, Rodeghier and Johnson (1993: 132) note, Hopkins, Jacobs and Westrum "apparently did not assess the reliability of this measure by using the customary formulas based on internal consistency, though it would be quite simple to do so." We applied these formulas to our data and achieved a reliability coefficient of .68 (coefficient alpha) for the abduction scale, which is quite acceptable for a five item scale. This, and other standard psychometric analyses we conducted, suggest that the scale is measuring a consistent phenomenon of some kind, but it tells us nothing about what it is that the scale is measuring. Table One
"UFO-Indicator" Experiences: National vs. South Jersey Samples
|Experiences:||National Sample||South Jersey Sample|
|1. "Waking up paralyzed with a sense of a strange person or presence or something else in the room.'||18%||25%|
|2. "Experiencing a period of time of an hour or more, in which you were apparently lost, but you could not remember why or where you had been."||13%||17%|
|3. "Feeling that you were actually flying through the air although you didn't know why or how."||10%||12%|
|4. "Seeing unusual lights or balls of light in a room without knowing what was causing them."||8%||13%|
|5. "Finding puzzling scars on your body and neither you nor anyone else remembering how you received them or where you got them."||8%||18%|
|"Abductee": Having four or five of the above experiences.||2%||3.4%|
Assessing the validity of a measure is much more difficult and is really the crux of the issue. By validity, we simply mean whether the measure measures what it is supposed to measure, in this case UFO abductions. The best method (known as criterion validity) is to correlate the measure with a known outcome measure. Thus, a measure of "scholastic ability" can be validated examining whether it correlates with school performance (other things being equal). This method cannot be used in this case, however, since we have no generally accepted measure of abduction status. The only viable alternative is to examine the measure's "construct validity." This means examining whether the measure performs as the theory predicts.
In this case, there are at least two alternative theories which can explain why the measure is internally consistent. One is that the respondents are consistently reporting on similar experiences as UFO abductees. The other is that the individuals who score high on the scale share a psychological tendency to have false memories. Flournoy (1911) referred to this phenomenon as cryptomnesia. Psychologist Robert Baker (1992: 78) states that this phenomenon of "seeing complex visual images in one's head that you cannot remember ever having seen before or...suddenly hearing voices from unknown and unrecollected sources is not only a much more common occurrence than is generally known but is also one of the more interesting and intriguing anomalies in the field of `normal' human behavior."
To distinguish between these two theories, we included three items in our questionnaire which asked about "unusual personal experiences" which are not part of the "UfO abduction" syndrome as posited by Hopkins, Jacobs and Westrum. These three items are numbers 11, 12 and 13 in Table Two. As Table Two shows, these items correlated (Pearson's r) with the "UFO Abduction" measure as well as the other items from Hopkins, Jacobs and Westrum's study which are also included in Table Two. Three of Hopkins, Jacobs and Westrum's items (numbers 6, 7, and 8) actually say nothing about UFOs, but can better be thought of as tapping a cryptomnesiac tendency to have "psychic" or "New Age" experiences (Ring, 1992). If all these thirteen items are combined in a single scale, we obtain a reliability coefficient of .75.
Items Correlated with "UFO-Indicator" Experiences
|6. "Feeling as if you left your body."||.30|
|7. "Having seen, either as a child or as an adult, a terrifying figure - which might have been a monster, a witch, a devil, or some other evil figure in your bedroom, closet or somewhere else."||.38|
|8. "Seeing a ghost."||.26|
|9. "Seeing a UFO?"||.21|
|10. "Having vivid dreams about UFOs."||.16|
|11. "Having a sudden feeling that something bad was happening to someone you know, and later finding out that you were correct."||.37|
|12. "Seeing a halo or ring of light around another person which other people couldn't see."||.23|
|13. "Being visited by a deceased relative or friend who spoke to you by name."||.31|
used a statistical technique known as factor analysis to
analyze how the responses to the thirteen items grouped
together empirically. The factor analysis found that a single
factor accounts for the consistency of the measure. There is
no empirical basis for separating the "UFO" items from the
items which do not relate to the abduction phenomenon. Table
Three shows the loadings on this principal factor. The
loadings measure how well each item correlated with the
general factor which the group of items have in common. As the
table shows, the three non-UFO related items (11, 12, and 13)
fitted right in with the other items which were intended to
measure UFO abduction experiences. An attempt to divide these
items into groupings through a technique known as "rotating
the factors" was not successful. The only items which
clustered together were "seeing a UFO" and "having vivid
dreams about UFO's" which are obviously similar. If we treat
these two items as belonging to a separate factor, the item
which is most closely related to them is "seeing a ghost." The
factor analysis strongly supports the view that these thirteen
items are a measure of cryptomnesia, not of UFO abduction. We
will henceforth refer to them as the cryptomnesia scale.
This conclusion is also strongly supported by Dawes and Mulford's (1993) innovative study at the University of Oregon which demonstrated that the dual nature of Hopkins, Jacobs and Westrum's first item, which asked about waking up paralyzed and about sensing a strange person in the room in the same item, actually led to an increased recollection of unusual phenomena as compared to a properly constructed single-issue survey item. Textbooks on questionnaire writing universally warn against "double-barreled" questions of this sort because they are known to give bad results. Dawes and Mulford confirm this and further offer the explanation that the combination of the two issues in one item causes a conjunction effect in memory which increases the likelihood of false recollection.
While the Hopkins, Jacobs and Westrum scale is not a valid measure of UFO abduction, they have inadvertently constructed a useful measure of another phenomenon: the tendency to have false memories. Therefore, we will henceforth refer to this measure as the cryptomnesia scale. Further support for this interpretation can be obtained by correlating the cryptomnesia scale with other items in the questionnaire. To the extent that the scale behaves as Baker's theory would predict, we can be more confident of its validity as a measure of cryptomnesia. Table Four shows correlations between cryptomnesia and a number of other questionnaire items. All the correlations reported in this study are statistically significant (p < .05).
The first three items show that high scores on the cryptomnesia scale are related to belief in conspiracies, including the JFK and AIDS conspiracies, as well as the UFO conspiracy. Cryptomnesiacs are also more likely to believe in astrology, and less likely to trust their neighbors, relatives and parents. They are more likely to have difficulty dealing with the stress in their lives, are more likely to wake up with worries at night, and to feel that people are conspiring against them. They are more likely to enjoy reading about UFOs and other strange phenomena. We did not, however, obtain significant correlations with items measuring anomie (a feeling of normlessness, or not really belonging to society).
Correlated with a High Frequency of "Unusual Personal
|14. "High government officials were involved in the Kennedy assassination."||.21|
|15. "The AIDS virus was created deliberately as part of a conspiracy against certain groups in our society."||.15|
|16. "The Air Force is hiding evidence that the United States has been visited by flying saucers."||.20|
|17. "I believe in astrology."||.16|
|18. "I can usually trust my neighbors."||-.16|
|19. "I can usually trust my relatives."||-.15|
|20. "When I was growing up, I could usually trust my parents."||-.12|
|21. "I often wake up in the middle of the night worrying about things."||.16|
|22. "I sometimes feel that people are conspiring against me."||.22|
|23. "I sometimes find it difficult to deal with all the stress in my life."||.16|
|24. "I enjoy reading books about UFOs and other strange phenomena."||.22|
of these findings are not surprising. David Jacobs (1992) has
noted high stress levels and frequency of psychosomatic
symptoms among "UFO abductees" which he attributes to the
trauma of the abduction experience. The question here is one
of distinguishing cause from effect - do the abductions cause
the stress, or does the stress cause the false memories which
are interpreted as abductions under hypnosis? It is difficult
to resolve this kind of cause and effect issue with data
gathered at one point in time. Since our analysis suggests
that we are dealing with cryptomnesia, not UFO abduction,
however, we undertook a causal path analysis based on the
assumption that this is a psychological phenomenon rooted in
early life experiences. The results of this analysis are
summarized in the path diagram.
The path analysis suggests that cryptomnesia is rooted in a lack of trusting relationships. This problem may have its origins in early childhood, but it continues into adult life with a lessened feeling of trust of friends and neighbors. This lack of trust leads to feelings of anomie and anxiety which make the individual more likely to construct false memories out of information stored in the unconscious mind. People who think in this way are susceptible to belief in conspiracy theories since these theories help them to make sense of an otherwise incoherent world.
This causal analysis is consistent with our data, but certainly not proven by it. Path analysis is not a substitute for in-depth psychological case studies. The path coefficients are modest in size, probably because most of the variables were measured with brief three-item scales because of the need to keep the questionnaire brief. This analysis does, however, provide a more plausible interpretation of the data than the UFO abduction hypothesis. It is also consistent with the results of detailed psychological testing on abductees (Slater, 1985; Klass, 1989).
Dr. David Jacobs was kind enough to speak to our class to familiarize the students with the issue, and I had the opportunity to speak with him informally after the lecture. At that time, I mentioned the "UFO abduction" case discussed in Siegel's (1992) book Fire in the Brain. Jacobs had absolutely no interest in learning of Siegel's findings, and expressed the view that no one was qualified to speak on this issue unless they had done dozens of interviews with abductees under hypnosis, as he had. He clearly fit the profile of the true believer as described in my book Turncoats and True Believers (1992). He used numerous ideological defense mechanisms to avoid confronting unwelcome evidence.
This closed mindedness can be observed in Jacobs' book Secret Life (1992). As the reviewer for the Journal of UFO Studies (Rodeghier, 1992: 186) observed: "Does Jacobs lead his witnesses? Sadly, one must answer in the affirmative." The whole weight of his argument in the book depends on hypnosis sessions which he conducted himself, and in which his strong convictions cannot help but influence the respondents.
The dogmatism of Jacobs and his associates has also been noted by others in the community of believers in UFO abductions. Abductee Karla Turner (1993: 26) has written that "it is a myth that alien abductions of humans follow a set pattern or agenda...David Jacobs...and other writers hold a diversity of intelligent, often ingenious theories, yet each makes the same error. They ignore parts of the abductions evidence--whatever details don't support their ideas." UFO investigators Stefula, Butler and Hansen (1993) confronted this dogmatism when their investigation of the prominent Linda Napolitano case uncovered serious flaws and apparent fabrications. When they shared this evidence with UFO experts including Budd Hopkins and David Jacobs, they found them completely unwilling to consider the evidence. Stefula, Butler and Hansen (1993: 20) were disappointed since:
The survey data promise some relief from this subjectivity, thanks to the objectivity of random sampling and statistical analysis. However, even when the data are collected and reported accurately as they apparently were in this case, the biases of the investigator can enter into the interpretation. In this case, only a strong belief in the prevalence of the abduction phenomenon led the researchers to interpret questions about seeing ghosts or having "out of body" experiences as indicators of abduction. Our data show that there is no objective basis for that assumption. The observed correlations are part of a broader psychological syndrome which includes non-UFO related phenomena. Of course, none of this proves that UFO abductions have not occurred in a small number of cases. In fact, two of our respondents volunteered that they had experienced such events. (We made no attempt to verify their accounts.) However, the public can rest assured that there is no evidence that millions of Americans are being abducted. Further research on the psychological phenomenon of cryptomnesia, however, is warranted. Both our survey and Hopkins, Jacobs and Westrum's representative national sample suggest that it is an identifiable syndrome affecting several million people in American society.
Robyn Dawes and Matthew Mulford, "Diagnoses of Alien Kidnappings That Result from Conjunction Effects in Memory," Skeptical Inquirer, 18 (1): 50-51, fall 1993.
Theodore Flournoy, Spiritism and Psychology, New York: Harper, 1911.
Ted Goertzel, Turncoats and True Believers, Buffalo, NY: Prometheus, 1992.
Robert Hall, Mark Rodeghier and Donald Johnson, "The Prevalence of Abductions: A Critical Look," Journal of UFO Studies, n.s. 4, 1992, pp. 131-135.
Budd Hopkins, David Jacobs and Ron Westrum, Unusual Personal Experiences: An Analysis of the Data from Three Major Surveys, Las Vegas: Bigelow Holding Corporation, 1992. This survey was replicated in 1998, with a decline in symptoms noted.
David Jacobs, Secret Life, New York: Simon & Schuster, 1992.
Philip Klass, UFO Abductions: A Dangerous Game, Buffalo,
Philip Klass, "Additional Comments About the `Unusual Personal Experiences' Survey," Skeptical Inquirer, Winter 1993, pp. 145-146.
Kenneth Ring, "The Cosmic Connection" (interview by Jonathan Adolph and Peggy Taylor), New Age Journal, July/August 1992, pp. 66-69, 122.
Mark Rodeghier, "Review of Secret Life by David Jacobs," Journal of UFO Studies, n.s. 4, 1992, pp. 184-189.
Elizabeth Slater, "Conclusions on Nine Psychologicals," in Final Report on the Psychological Testing of UFO "Abductees," Mount Rainer, MD: Fund for UFO Research, 1985.
Joseph Stefula, Richard Butler and George Hansen, "Budd Hopkins Case of the Abduction of Linda Napolitano," photocopied, January 8, 1993.
Ronald Siegel, Fire in the Brain, New York: Dutton, 1992.
Lloyd Stires, "Critiquing the `Unusual Personal Experiences' Survey," Skeptical Inquirer, Winter 1993, pp. 142-144.
Turner, "Encounter Phenomena Defy `Set Pattern'," UFO
Magazine, Vol 8, No. 1, 1993, pp. 26-30.
A story about this research appeared in the National Enquirer, completely reversing the findings. They said that "top university researchers" (i.e., the Methods and Techniques of Social Research class at Rutgers in Camden, had done the study, then they quoted Ron Estrum of Eastern Michigan University, a believer in UFO abduction, as saying what the results were: