Analysis of the SAFE poll on fossil collecting
Jeff Poling
INTRODUCTION
Late last year a poll was taken by MKTG, Inc., on behalf of the "paleontological community," regarding public opinion ostensibly about commercial collecting of fossils on public lands. The poll is available on the Society of Vertebrate Paleontologists website under the auspices of Save America's Fossils for Everyone (SAFE). The results of this poll have been trumpeted by those opposed to proposed federal legislation that would allow commercial collecting on federal land. SAFE states:
- "Based on the results of this poll, the American public are overwhelmingly against commercial collecting on Federal public lands. Any legislation that enfranchises commercial collecting of fossils on Federal public lands, such as the pending 'Fossil Preservation Act of 1996' introduced by Representative (D., S. Dakota) Tim Johnson, would be counter to public opinion."
The poll is also used by those who want legislation specifically forbidding any sort of collecting, whether commercial or private on public or private land, stating that the poll proves the public desires such restrictions.
I contend that the poll proves neither of these conclusions.
ANALYSIS
To start, we need to determine the statistical parameters that govern this poll.
| First, is the sample size sufficient?1 We assume that the population proportion (percentage of people responding a certain way) should be within plus or minus 7 percentage points of the sample proportion, the stated accuracy of the poll. Calculating the sample size, at right, we see the sample size is sufficient for an interval of plus or minus 7 percentage points. |  |
| Second, what are the results of the hypothesis test?2 The null hypothesis will be that the population proportion is equal to or greater than 51%, making it the majority opinion. The alternate hypothesis will be that the population proportion is less than 51%. The Type I error coefficient is set to 5%, meaning we want to risk only a 5% chance of rejecting a question's response as the majority opinion when it really is. Running the hypothesis test, at right, we see that any response that was given by at least 46.26% of the respondents in the sampling can be considered the population's majority opinion. |  |
| Lastly, what is the probability of making a Type II error, accepting a response as the majority opinion when it is not?3 The graph, at right, shows the probability of making a Type II error when the population proportion is as shown. For example, there is approximately a 60% chance of accepting a response in the sampling as the population's majority opinion when only 47%, or 4% less than the majority, of the population holds that opinion. |  |
In general, MKTG, Inc., appears to have its statistics right. However, as will be shown, whether the questions were appropriate and whether SAFE's interpretations are correct, is doubtful. The text between the rules, below, is the text and results of the SAFE poll. Text in italics is my analysis of the questions and did not appear in the original poll.
Poll scenario #1:
Imagine, that you have inherited a large ranch out West. On a visit to your ranch you discover the fossil bones of an animal. At first you think that they are the bones of a cow that died in recent years. However, at closer inspection you find that the bones are stone, the skull is strange-looking, and the backbone looks different from anything you've seen. Pieces of bone are washing out of a rock ledge, they are falling apart and appear very fragile. You recall that someone told you that the fossil bones of ancient creatures, millions of years old are sometimes found in the area. Remember you now own the ranch.
Poll scenario #2:
After your discovery you obtain a detailed survey of your property, you find out that you had actually wandered off of your property and into Public Property part of a National Grasslands, a Federal Wilderness area or a National Park. The fossil is not on your lands but rather on these Public Lands.
Poll scenario #3:
Imagine, you are on another visit to the ranch. Again you wander off of the ranch into Public Property part of a National Grasslands, a Federal Wilderness area or a National Park. This time you discover the fossils of animals without backbones. Some look like crabs, some like corals and other unlike anything you have ever seen. The rock seems loaded with their impressions. You recall that someone told you that the fossils of these strange creatures, millions of years old, are sometimes found in the area. Remember you are on Public Property.
| Scenario #1, Vertebrate -- Private | Scenario #2, Vertebrate -- Public | Scenario #3, Invertebrate -- Public | |
|---|---|---|---|
| (Q1) The fossil is mine, finders keepers | |||
| Agree | 46.0% | 17.0% | 16.3% |
| Disagree | 48.7% | 81.0% | 81.3% |
(Q2) The fossil could be of scientific importance, I should report it to appropriate scientific authorities | |||
| Agree | 90.3% | 97.4% | 96.0% |
| Disagree | 7.6% | 2.4% | 3.0% |
| Is there a difference between reporting the find and having the find taken away? I think there is, and I wonder what the response would be if the question was stated to indicate the fossil should be reported to "appropriate scientific authorities," whatever that means, or be prosecuted. | |||
(Q3) The fossil could be of scientific importance, if they want it I should allow a museum or university to collect it. | |||
| Agree | 83.7% | 90.7% | 90.7% |
| Disagree | 13.7% | 7.0% | 5.3% |
| "I should allow a museum or university to collect it." What would the response have been if the question asked whether the individual should be forced to allow a museum or university to collect it? | |||
(Q4) The fossil is part of our heritage, it belongs to everyone in the United States. | |||
| Agree | 68.0% | 86.6% | 89.7% |
| Disagree | 26.7% | 12.3% | 8.0% |
| Is this an opinion of whether the private owner should make it available for people to see, or if the government should actually own the fossil. This might not be clear in everyone's mind. | |||
(Q5) It's within the bounds of my property, I should be allowed to do whatever I want to do with it. What's important here are my property rights. | |||
| Agree | 63.0% | ||
| Disagree | 32.7% | ||
| Overwhelmingly, the respondents feel that the fossil belongs to the property owner to do with as he wishes. This is in direct contradiction to the results of Q1. | |||
(Q6) There should be a law prohibiting my taking the fossil out of the ground. | |||
| Agree | 37.3% | 70.7% | 73.6% |
| Disagree | 56.7% | 25.4% | 21.3% |
| Note that, again, private property rights are to be respected. | |||
(Q7) There should be a law against my selling the fossil. | |||
| Agree | 36.6% | 80.0% | 81.7% |
| Disagree | 58.0% | 17.3% | 15.7% |
| Note that, again, private property rights are to be respected. | |||
(Q8) There should be a law against my taking the fossil out of the United States. | |||
| Agree | 57.0% | 81.0% | 84.3% |
| Disagree | 37.3% | 13.3% | 13.0% |
| I'm not sure what to make of this one. The United States regularly exports other items, why are fossils singled out? Further, important specimens find their way into domestic museums via fossil import from countries where fossils are not a highly regarded scientific object. Would such a prohibition prevent such an exchange? | |||
| (G1) It's okay with me for someone to buy and sell fossils. | |
| Agree | 49.3% |
| Disagree | 42.6% |
| Respondants agree that it's okay for people to sell fossils. This is in direct contradiction of Q7 in my opinion: if selling of fossils is okay, why have a law against it? This is either a contradiction, or evidence of a rather frightening mindset in modern Americans. | |
(G1a) It's okay with me for someone to buy and sell common fossils. | |
| Agree | 65.3% |
| Disagree | 25.5% |
(G1b) It's okay with me for someone to buy and sell rare fossils, perhaps scientifically important fossils. | |
| Agree | 34.0% |
| Disagree | 62.1% |
| Its not okay with the respondents that rare fossils be sold. Note that this would prohibit the sale of ALL rare fossils, whether from public or private land. Is this a contradiction of previous responses? | |
(G2) Fossils found on Public Lands should be restricted. It should be illegal to collect them, to sell them, to destroy them, to export them out of the United States. | |
| Agree | 80.3% |
| Disagree | 14.3% |
| This law, as stated, would prevent even professionals from collecting them, and would prevent scientific exchange between American and foreign institutions. Not good...... | |
(G3) Fossils found on private land should be legally available for sale. | |
| Agree | 46.0% |
| Disagree | 48.0% |
| This contradicts responses to Q5, Q6, Q7, G1, G1a and G8. | |
(G4) All fossils found in the United States, whether found on private or public lands should be the property of public institutions like museums or universities. | |
| Agree | 64.0% |
| Disagree | 31.6% |
| This contradicts Q5 and Q7. | |
(G5) There should be a law to stop people from collecting fossils on Federally Managed Public Lands. | |
| Agree | 75.7% |
| Disagree | 19.3% |
| This is basically a restatement of G2, and is pretty much in agreement with the results. | |
(G6) There should be a law to stop people from collecting fossils on all state lands. | |
| Agree | 64.7% |
| Disagree | 27.0% |
| Does this restrict professionals from collecting on state lands, or are paleontologists not considered people. | |
(G7a) Fossils of animals with backbones are part of our national heritage and should be protected in much the same way that Archaeological remains (human artifacts) are now protected. | |
| Agree | 85.3% |
| Disagree | 7.4% |
(G7b) Fossils of animals without backbones are part of our national heritage and should be protected in much the same way that Archaeological remains (human artifacts) are now protected. | |
| Agree | 82.7% |
| Disagree | 9.7% |
| Do the respondents even know how archaeological remains are now protected? | |
(G8) This is the United States, we should encourage free enterprise. A law restricting the selling of fossils collected on private lands is wrong. | |
| Agree | 54.7% |
| Disagree | 38.0% |
| Yet another question whose response is in favor of private ownership. | |
(G9) If someone finds a fossil of a dinosaur and wants to keep it in their basement that's fine with me. | |
| Agree | 31.0% |
| Disagree | 62.7% |
| Does that mean, though, that the individual should make the fossil available to see, should find a better place to keep it, or should have it wrested away? | |
(G10) If someone finds a fossil of a dinosaur they should not remove it unless they obtain the aid of professionals/scientists. | |
| Agree | 89.7% |
| Disagree | 6.0% |
| Is this an opinion that the discoverer should seek the aid of a professional, or legally must seek aid? | |
(G11) If laws are created to restrict the collection of fossils on Public Lands, the only people who should be allowed to collect them are people with appropriate skills for doing so and with a permit for that purpose. All the fossils that they find should go into museums or universities prepared to protect them. | |
| Agree | 88.0% |
| Disagree | 9.0% |
| This question not only would prohibit commercial collecting, but private collecting as well. | |
(G12) Fossils bring big money these days, they should be allowed for sale just like any other commodity. | |
| Agree | 32.6% |
| Disagree | 59.6% |
| This contradicts responses to Q5, Q6, Q7, G1, G1a and G8 as it pertains to privately collected fossils. | |
This statement must refer to a poll other than the one above. Responses to questions that unambiguously single out personal property rights regarding fossils are four to two in favor of private property rights.
Many of the questions could have meant different things to different people. For example, what did people really mean when they said a dinosaur fossil should not be kept in someone's basement? There are many possible responses depending upon what the person thought. Further, how many of them remembered, or were even told, that this was a poll geared toward determining public opinion on fossil related legislation? How would responses have been different if it had been made clear in the question that the law would step into the situation presented, such as the government invading the person's home who keeps a dinosaur fossil in the basement and taking that fossil away? By not spelling out these issues, the responses are suspect. Certainly the interpretations of the results SAFE and others seem to be making are suspect. It appears they are interpreting such things as "I should allow a museum or university to collect it" to mean "fossil finds should be collected and owned only by the 'proper authorities' by force of law," a leap of logic to say the least. Certainly, the poll does not show Americans clearly favor restrictions on collecting or selling fossils found on private land, as many people are saying.
Whether this poll shows Americans favor restrictions on commercial collecting on public land is not clear. All three of the example scenarios involve an individual, not a company. One can infer that negative responses toward individuals collecting on public lands also extend toward commercial collectors; however, if the respondant knew that the commercial collector was equally competent at collecting a fossil as paleontological professionals, and would make scientifically important fossils available to the scientific community, would their responses have been different? Further, one can infer that opposition to sale of fossils means opposition, by definition, of commercial collection of fossils; however, the results of questions focusing on the issue of fossil sales are contradictory and therefore do not give clear signals as to public opinion.
The poll also asked some meaningless questions about general education, income, hobbies, etc. Only one question, whether they liked to collect fossils (4%), had any bearing on the issue at hand. What the questions didn't ask was whether the respondents knew what they were talking about. How many of them knew the issues regarding fossils, including how and where they are preserved, how and where they are collected, the effects of erosion on fossils, the state of paleontology as a science, the number of scientists actually collecting, the ability of museums to collect, store and maintain fossils, how many important finds were due to private and commercial collection, and other important issues. What would the responses of a person who knew these issues have been? Had the issues been explained to the respondents, would their responses have been different?
One should also note that the SAFE poll was as unbiased as this article, which is to say not at all. SAFE states, regarding the Fossil Preservation Act of 1996, "Notice that the title of the Act is a misnomer. Fossils are not protected under this Act." Hardly unbiased, and, as far as I'm concerned, an outright lie. Polls can be very expensive, and political polls such as this one are rarely conducted in a way that allows conclusions contrary to what the sponsor of the poll wishes.4 What is particularly frightening is the preponderance of questions regarding fossils on private property; not only does SAFE, and, likely, the SVP, want to prevent collection of fossils from public land, they also want to take away the right of property owners to collect and sell fossils from their own land. Certainly, SAFE's interpretation of the poll shows clear bias, stating conclusions that are not supported by the poll.
CONCLUSION
Many people are trumpeting this poll as proving that public opinion is overwhelmingly in favor of legal restrictions on fossils, whether the fossils were found on public or private land, and whether the finder is a professional, commercial employee, or an individual. The poll was likely biased and, worse, made no attempt to distinguish whether the respondents understood the issues at hand, thus making the wisdom of following their opinions suspect. Regardless, the poll does not prove public opinion is in favor of legal restrictions as 1) many results were contradictory, 2) results were clearly in favor of personal property rights despite claims to the contrary, and 3) questions were not worded in such a way as to allow only a single or clear conclusion.
FOOTNOTES:
- David R. Anderson, Dennis J. Sweeney and Thomas A. Williams, Statistics for Business and Economics (St. Paul: West Publishing Company, 1993), 781-783.
- Ibid., 313-317.
- Ibid., 320-322.
- Dr. Glenn W. Milligan, MBA 870 Statistical Analysis for Decision Making class lecture, The Ohio State University Columbus campus, Autumn 1993.
Copyright © 1996 by Jeff Poling
Revised November 11, 1996