Identifying 8 necessary criteria for a work to be considered useful science:
March 1, 2017's March for Science calls for “robustly funded” science and “political leaders and policymakers to enact evidence-based policies in the public interest.” But is this just an attempt to dress up the marchers’ political beliefs as science? And what do they mean by science?
Fortunately for those who care, there is a remarkable level of agreement in the writings of scientific pioneers such as Francis Bacon, Isaac Newton, and Benjamin Franklin on the nature of the scientific method. That agreement is also reflected in the definition provided by the Oxford English Dictionary.
We have expanded on the established definition and identified eight necessary criteria for a work to be considered useful science. The criteria include objectivity and full disclosure. We expect that most scientists would agree with these criteria as obviously true and important.
The pioneers of science charted the way by describing how to comply with the criteria. To be objective, according to Newton, the study should compare all reasonable hypotheses by using a fair and balanced experimental design.
We have summarized the eight criteria on a one-page checklist (available at guidelinesforscience.com ). You can easily refer to it to assess whether something you are looking at is a work of science. By using the checklist, you do not have to depend on an authority to tell you “this is what the science says.” Knowing and agreeing with the criteria in the checklist does not help. To be useful, the checklist must be used.
The checklist is concerned only with the scientific method, so one does not need to be an expert in the field or topic to use it. In fact, experts may have difficulty rating the scientific compliance of works in their own field. They are likely to be biased against findings that challenge conventional wisdom.
We found that the ratings of raters who did not use the checklist were unreliable. Their ratings differed substantially from those derived using the checklist. When faculty and students raters used the checklist, their ratings were remarkably consistent. The checklist is available at guidelinesforscience.com .
The checklist is badly needed. One cannot rely on the fact that a purported discovery was published in a high-status scientific publication. When we used the checklist to rate papers published in leading scientific journals, we found less than one percent of them to be compliant with the scientific method.
We suggest that you try out the checklist at the March for Science rallies. Show your respect for the scientific method and, as Newton emphasized, be willing to consider alternatives. Be fair in evaluating alternative hypotheses. You have to ask yourself the question, “Can I imagine any evidence that would prove my favoured hypothesis is wrong?” If you can’t, you are not approaching the subject with an open mind. You also fail Newton’s criteria for understanding science.
Speakers should comply with science. Listeners should be respectful and request of the speakers, “Please show us that you have complied with science.” It’s not enough for them to say that they have followed the scientific method.
Progress in all fields relies on the scientific method. Science is a never-ending, never-settled process.
Voting has no place in science. Scientific laws always eventually prevail over the political laws created by our elected officials.
The March for Science should not simply be another way for us to express our opinions. It should not be an effort to pressure scientists and voters to agree with us. The scientific method is the best way we have of engaging in factual disputes.
Written by:
J. Scott Armstrong (jscottarmstrong@upenn.edu ) is a professor at the Wharton School at the University of Pennsylvania. Kesten C. Green (kesten.green@unisa.edu.au) teaches managerial economics in the University of South Australia Business School.
J. Scott Armstrong (jscottarmstrong@upenn.edu ) is a professor at the Wharton School at the University of Pennsylvania. Kesten C. Green (kesten.green@unisa.edu.au) teaches managerial economics in the University of South Australia Business School.
