P-hacking: its implication for science and scientific research

doi:10.15406/aovs.2018.08.00282

Advances in

eISSN: 2377-4290

Ophthalmology & Visual System

Opinion Volume 8 Issue 2

P-hacking: its implication for science and scientific research

Andrews Nartey

Verify Captcha

Regret for the inconvenience: we are taking measures to prevent fraudulent form submissions by extractors and page crawlers. Please type the correct Captcha word to see email ID.

Department of Optometry and Visual Science, Kwame Nkrumah University of Science and Technology, Ghana

Correspondence: Andrews Nartey, Department of Optometry and Visual Science, Kwame Nkrumah University of Science and Technology, PMB, Kumasi, Ghana, Tel 233 50 137 4148

Received: March 26, 2018 | Published: April 20, 2018

Citation: Nartey A. P-hacking: its implication for science and scientific research. Adv Ophthalmol Vis Syst. 2018;8(2):115-116. DOI: 10.15406/aovs.2018.08.00282

Download PDF

Abstract

The globe has seen tremendous scientific inventions and technological advancements over the years. Inventions do not happen haphazardly but rather, it takes painstaking and rigorous application of scientific principles, bouncing ideas off one another, and the carrying out of experimental research studies. A number of indicators have been established to assess the precision of research findings. One of such notable indicators is the p-value, which denotes the statistical significance of a research finding. The manipulation and/or tweaking of research variables to obtain significant p-values have become an issue that have attracted considerable media interest and attention. This article serves to provide, in no uncertain terms, the researcher’s perspective in relation to the manipulation using p-values, whilst reviewing related literature. The paper concludes with the implication this phenomenon has for science and scientific research.

Keywords: P-value, research, science, inventions, manipulations

Opinion

In a rapidly evolving scientifically and technologically savvy world, the globe never ceases to be inundated with revolutionary inventions and groundbreaking technology. The only way these inventions and discoveries would become endorsed and consequently patronized is by ascertaining their authenticity and validity through experimental research studies. Experimental research studies, thus, have become the fulcrum around which new interventions get to justify their inclusion through what I would metaphorically term as “preliminary stage competitions, following which successful experimental teams become seeded.” So yes, scientists are now in competition or if you like, an intellectual rush to etch their names and products in the annals of scientific laurels. This arguably has led to the concept of what is now popularly known as “p-hacking.”

P-hacking simply refers to the wrongful manipulation of research methodologies, data, analyses and findings to serve a particular interest – validate a given hypothesis. One can situate p-hacking in the context where an astute and sagacious scientist and/or researcher sits behind his computer desk and indulges in the trial and error business of fine-tuning various research elements and observations till these elements culminate into statistically significant results. And the conduit for such ill manipulations lies in the use of the overprized p-value. The p-value is used to denote the statistical relevance of a research finding and is usually set at 0.05 in the biosciences; so that any p-value obtained which is equal to or less than this value makes the said finding statistically significant and not due to random chance or a research operational flaw. But the big questions to explore are: Is the p-value the only indicator of the strength of evidence? And do scientists p-hack deliberately or it just happens unconsciously? Should p-hacking be considered one of those normal occurrences or is it an incriminating evidence of academic fraud or dishonesty?

Simonsohn et al.¹ theorised that p-hacking is necessitated by the natural human tendencies to bias, and some biases are indeed helpful. He added that, this is purely “human nature” – so people p-hack unconsciously and therefore p-hacking should, in no way, be seen as a conscientious effort on the part of scientists to promote themselves and their products. According to Simonsohn,² “when the first analysis you try doesn’t spit out the result you want, you keep trying until you find one that does.” This however defeats his stance of unconscious p-hacking. John Ioannidis³ mentioned that humans are biased to finding exciting results and admitted that most research findings couldn’t be taken at face value due to potential biases and errors that emanate from manipulations such as p-hacking. Brian Nosek et al.⁴ opined that, in as much as p-hacking is generally seen as wrong, it does help in the exploration and advancement of knowledge, and should thus, not be discouraged. Meanwhile, other scientists have justified this conduct by adding that, science has a mechanism of self-correction, and so untrue scientific conclusions can be retracted when they are proven to be wrong.

So it is very apparent that there is some underlying consciousness to p-hacking from the few commentaries outlined above. For a researcher to have some preconceived finding for a particular research experimentation is one thing but to allow these preconceptions to supplant sound ethical judgements is another thing, which should not be a tad, encouraged. The concept of retraction of scientific papers as a justification for the tolerability of scientific flaws would only undermine the meticulousness, rigor, and objectivity of science and its players. The overarching goal of all scientific experimental researches is to either validate or invalidate a given hypothesis. And this goal should be held in very high esteem and not sacrificed on the altar of unmerited scientific eminence. It is arguably true that the p-value has become overrated in scientific researches, and it is worth mentioning that the p-value is not the only measure of evidence of precision and repeatability of scientific researches. Most importantly, a precise research finding does not equate to validity or truth since bias and confounders could sway research results in a particular direction. This only brings to fore, the need for comprehensiveness and meticulousness in the way we conduct our researches and analyze our research data.

P-hacking would only lead to the churning out of tonnes of research papers, majority of which would be of lower quality: such studies when p-hacked cannot be replicated Tackett et al.⁵ In addition, some valuable information from research could be easily concealed from the public if it does not seek to champion the cause for which a researcher undertook that study. It is worth noting that, if the rate of p-hacking is not curbed, the public’s confidence in science and scientific findings would keep dwindling; and this would render scientific researches largely needless. So it behoves all scientists to strive hard to keep science as science, and not as business or politics. The Latin aphorism “scientia potentiaest” which translates as “knowledge is power” should therefore not be taken for granted. Science is indeed a powerful instrument for social change and economic development, and it is recommended that, the standards in the application of this scientific knowledge should be kept as high as possible.