Out Of Darkness
Fundamentalist Religion and Extraterrestrial Intelligence – Part I
Introduction
This is the first of a multi-part study regarding the relationships between extraterrestrial life and fundamentalist religion. The initial three blog articles will consider five different surveys of the possible impact that certain knowledge of alien civilization would have upon organized religion. I will suggest that many of these surveys are seriously flawed: biases abound in the sample size and selection, in the questions themselves, and in the interpretation of the responses. I will also argue that the response interpretation may be crippled by a core misunderstanding of the fundamentalist religious mindset. While I will present the basics of these surveys for your review, I recommend obtaining a copy of all of these papers to make your own assessment. After the survey articles, I will discuss several published papers on the topics of extraterrestrial intelligence and religion. There is enough material here for a book (or several); in fact, several books have been written on this subject.
What kicked this series of blogs off was the latest issue of the Journal of Scientific Exploration. To be up front on my own perspective, I was brought up fundamentalist Southern Baptist. I know all the “lingo” and attitudes of a fundamentalist Christian. I know how these people think and I am familiar with how they react towards different subjects.
Revisiting the Alexander UFO Religious Crisis Survey (AUFORCS): Is There Really a Crisis?
The summer issue of the Journal of Scientific Exploration (JSE) has several interesting articles, one of which is “Revisiting the Alexander UFO Religious Crisis Survey (AUFORCS): Is There Really a Crisis?” written by Jeff Levin [1]. Jeff Levin is associated with the Institute for Studies of Religion, Baylor University, Waco, Texas. The JSE article reviews the AUFORCS survey and results.
Purpose of the survey
Would disclosure of information regarding prior contact with extraterrestrials precipitate a religious crisis?
A private study of 229 Protestant, Roman Catholic, and Jewish clergy was conducted in 1994. There were eleven questions and a five point Likert scale was used. The response categories were “Strongly Disagree,” “Disagree,” “Neither Agree or Disagree,” “Agree,” and “Strongly Agree.” Levin was able to obtain the AUFORCS data archive for review for the JSE article.
AUFORCS Survey Method
· The survey was conducted by mail.
· Randomness was obtained by using potential respondents from Data Base American Companies’ PhoneDisc Reverse Fall 1993 database.
· Surveys were sent to 1000 clergy.
Breakdown of participants:
563 Protestant churches -- 133 actual respondents
396 Roman Catholic churches -- 86 actual respondents
41 synagogues – 10 actual respondents
45 surveys were returned because of incorrect addresses. 230 “answered” surveys were returned and 229 were used in the analysis. One survey was rejected due to technical ambiguities according to the author.
The JSE article admits that the 24% response rate in the Alexander survey is low according to other national probability surveys. Levin writes that this low response rate is offset by the importance of the survey. He states this survey was a first ever look at a social issue of critical importance and it also had historical importance. Does this trump scientific accuracy? Is it true that the social importance of the survey outweighs any bad analysis done with potentially biased data? In this case, the data appears to be dangerously biased.
A 24% response rate should raise all kinds of red flags when conducting any kind of data analysis. Some statisticians have stated that anything below a 50% response rate could be considered biased data and should be discarded altogether. In general, the lower the response rate is, the less credible the results will be in the analysis. I quote, “Response rates are generally considered to be the most widely compared statistic for judging the quality of surveys.” (Biemer and Lyberg, 2003) [2]. According to the Instructional Assessment Resources page from the University of Texas at Austin, acceptable response rates vary by how the survey is administered [3]. For a mail survey 50% is considered adequate; 60% is good; and 70% is very good. The Journal of the American Medical Association has an editorial regarding minimum response rates for survey research [4]. It states that some biomedical journals require a 60% minimum response rate for surveys. It has been shown that bias decreases as the response rates increase, so researchers want to see a higher response rate with their surveys to reduce bias in the survey data. Some survey researchers will argue that the response rate is not the only consideration for judging survey quality, but it is a very important one. The Office of Quality Improvement from the University of Wisconsin-Madison has a paper entitled, “Survey Fundamentals – A Guide to Designing and Implementing Surveys.” On page 14 the document states, “The UW Survey Center normally receives a 60-70% rate of response to mailed surveys. For web surveys, a 30-40% response rate is common.” [5] The US Government document “Office of Management and Budget Standards and Guidelines for Statistical Surveys” [6] indicates such a concern with bias in surveys that if a survey has a response rate of less than 80%, further analysis must be conducted to evaluate what is considered the nonresponse bias. Statistics books discuss the concept of nonresponse bias and this is a topic I will leave for the reader to research.
So, right off the mark, there is an extremely low response rate of 24% in the AUFORCS survey, and the response rate is ignored and the statistical analysis continues. Danger Will Robinson, we are treading on wafer thin ground here.
One item I found particularly interesting in the JSE article was the number of completely blank questionnaires returned to the investigator which contained lengthy comments and, in the author’s description, lengthy sermonettes. According to Levin, these comments expressed disapproval of the topic, Bible verses were stapled to the surveys, concern was expressed about the researcher being involved in this topic, etc. Think of the motivation it takes for someone to take the time to write out admonishments, staple Bible verses, write sermons and so forth on a survey and mail it back. It is difficult enough to get people to respond to surveys in general and some of the clergy selected for the survey not only did not participate in the survey, but used it to deliver a sermon to the researcher. Does this behavior by the selected sample of participants raise any red flags with anyone? Does this conduct perhaps speak to the nonresponse survey bias already noticed in the extremely low response rate? Does this already speak to the survey’s basic premise about investigating a possible crisis in religion due to contact with extraterrestrials? Who responds to surveys in this manner? Who is so motivated to take the time to respond to a survey by “preaching?” What kind of individual does this describe? What does this say about them?
The following are the eleven survey questions from the Victoria Alexander UFO Religious Crisis Survey:
1. “Official confirmation of the discovery of an advanced, technologically superior extraterrestrial civilization would have severe negative effects on the country’s moral, social, and religious foundations.”
2. “My congregation would perceive any contact made with a technologically advanced extraterrestrial civilization, direct or indirect, as a threat.”
3. “The discovery of another intelligent civilization would cause my congregation to question their fundamental concepts regarding the origin of life.”
4. “If highly advanced intelligent civilizations exists [sic] elsewhere in the universe, the basic tenets of religion would be present.”
5. “Genetic similarities between mankind and an advanced extraterrestrial civilization would challenge the basic religious concepts of man’s relative position in the universe.”
6. “If an advanced extraterrestrial civilization had religious beliefs fundamentally different from ours, it would endanger organized religion in this country.”
7. “Scientific confirmation of contact with an advanced extraterrestrial civilization is probable in our lifetime.”
8. “It is unlikely that direct contact with an advanced extraterrestrial civilization has occurred or is currently ongoing.”
9. “My congregation would question their beliefs if an advanced extraterrestrial civilization had no system of religion.”
10. “If an advanced extraterrestrial civilization proclaimed responsibility for producing human life, it would cause a religious crisis.”
11. “I believe my answers to the preceding questions represent the views of my congregation.”
As a test, I decided to answer these questions based on a religious mindset. I can do that because I was raised fundamentalist Southern Baptist. I was also exposed to many other Protestant denominations as I was growing up. I submit that another fundamental flaw in this survey is the researchers’ failure to understand how a fundamentalist Christian would think about these questions. The religious world is very different from the scientific and general academic world. Without looking at the answers, I was able to answer all eleven questions correctly according to the survey results. I am not surprised at all that the clergy that did respond answered the way they did.
Take for example question 1. “Official confirmation of the discovery of an advanced, technologically superior extraterrestrial civilization would have severe negative effects on the country’s moral, social, and religious foundations.”
My predicted response: Of course not. A superior extraterrestrial civilization doesn’t exist. It is not possible. It could never be possible; therefore it could not ever happen. So of course, there will be no negative effects on my beliefs at all – ever; period. So the predicted response is strongly disagree. Duh.
James F. Strange’s paper “Some Observations from Archaeology and Religious Studies on ETI” considers the results from the Alexander Crisis Survey [7]. Strange combined the Strongly Disagree and Disagree category answers for all questions into one category, which totaled 77% for Question 1. Agree and Strongly Agree combined for 8% and Neither Agree or Disagree was 14%. Bingo. The rest of the questions were extremely easy to answer according to the survey. I felt like I was back in the pew listening to a hellfire and brimstone sermon and I answered all the questions very quickly. Easily done when you know how the religious folk think.
One glaring question needed to be in this survey. It would have helped clarify the survey’s results, such as they were. Do you believe in the possibility of extraterrestrial life? That question should have been in there and it wasn’t. Wonder why?
I believe that the statistical value of the Alexander UFO Religious Crisis Survey is minimal. I do not believe the F and p statistics mentioned in the JSE article have any significance in regards to the survey data. This is because I believe that there is tremendous bias in this and other influential surveys on this subject. I believe these results should not be taken to conclude that a disclosure of an alien presence would not precipitate a religious crisis. I have looked clergy in the eye and have had discussions about this very topic. I know the looks on their faces. I know how they react. I know what they say. They can’t comprehend it. It is all demons to them and nothing else. Of course their faith is strong; nothing can shake it when they know they have all the answers to what is possible and what isn’t.
Part II will cover the Peters ETI Religious Crisis Survey
References
[1] J. Levin, “Revisiting the Alexander UFO Religious Crisis Survey (AUFORCS): Is There Really a Crisis?”, Journal of Scientific Exploration, Vol. 26, No. 2, Summer 2012, pp 273-284.
[2] T. Johnson, L. Owens, “Survey Response Rate Reporting in the Professional Literature”, American Association for Public Opinion Research, Survey Research Laboratory, University of Illinois at Chicago, 2003
[3] Instructional Assessment Resources, University of Texan at Austin. http://www.utexas.edu/academic/ctl/assessment/iar/teaching/gather/method/survey-Response.php
[4] E. H. Livingston M.D., J. S. Wislar MS, “Minimum Response Rates for Survey Research”, JAMA Network, Feb. 2012
[5] Survey Fundamentals – A Guide to Designing and Implementing Surveys, Office of Quality Improvement, University of Wisconsin-Madison, www.quality.wisc.edu
[6] United States Government, Office of Management and Budget Standards and Guidelines for Statistical Surveys, 2006
http://www.whitehouse.gov/sites/default/files/omb/inforeg/statpolicy/standards_stat_surveys.pdf
[7] J. F. Strange, “Some Observations from Archaeology and Religious Studies on ETI”, University of South Florida, The Society for Planetary SETI Research
http://spsr.utsi.edu/articles/someobservations.html |
