Understanding the World: The Realities of Scientific Progress

Examining the Complexities and Biases of Scientific Research

Author's Preface

My Sketchy Background in Science

I did train in a field that some consider scientific, that is, experimental psychology, decades ago, I also worked as a research assistant in psychology laboratories for a number of years. I never became a working scientist. I went to graduate school and eventually decided that wasn't for me and went off to work in the computing information systems field. Nevertheless, I've kept a strong interest in science over the years, a lifelong interest with a lot of reading of works for the layperson, and I have become a critic of science and a champion of good science.

Science as the Best Method

I think it's still probably the best method we have overall for helping us better understand reality. However, it's a human enterprise, and it's very flawed, more flawed than I originally believed.

I was brought up to have an idealized picture of how science worked and how much it could achieve. It took me a long time to understand that that was a cartoonish picture. So, I have recorded my thoughts here.

I've had ChatGPT produce an introduction, a summary, a reference section, and some headings to segment my thoughts appropriately and produce citations.

And Sometimes We Just Flip a Coin

My most pressing concern today is the evaluation of health advice: treatments, diets, medication and so on. There are so many contradictory views on such things that I am at a loss as to what to do. When experts disagree, what is the layman to conclude? Will science eventually result in more accurate knowledge? I fear that success in this will come too late to make a personal difference. In the meantime, I can always flip a coin.

Introduction

The conventional view of how scientific progress works is often oversimplified. In reality, the process is much more complex and fraught with biases, missed opportunities, and barriers to publication. This essay breaks down the key factors that complicate scientific advancement, emphasizing the challenges researchers face in getting their work acknowledged and accepted.

There's a cartoonish picture of how scientific progress is. It's a naive view, cartoonish, simplistic. It does not reflect reality (Latour, 1987).

The Oversimplified Process of Publication and Acceptance

The view is that research is published and the scientific community looks at the research and evaluates it for quality and incorporates its findings or fails to incorporate its findings because they reject the findings (Ioannidis, 2005).

The truth is much more than that. First of all, research may be conducted and never published. Or it may be published in journals that are generally considered second-rate, pay-for-publish journals, minor journals with very little circulation (García et al., 2021).

So there may be research that gets published, but there may be a lot of research that doesn't get published for various reasons, rejected by peer reviewers. There's a lot of research that doesn't get funded, doesn't make it past the grant stage (Ioannidis, 2005).

So that means evidence is restricted. It might have been good evidence, it might have been poor evidence, but it's never published (McCook, 2018).

How Much Published Research is Encountered

And of that which is published, even within one field, even within a narrow specialty, scientists may not read it, may not encounter it (Landhuis, 2016).

Biases Against Researchers and Paradigms

Scientists may be biased from the get-go against a particular researcher or a particular research paradigm by their peers, by their colleagues, and fail to give the publication proper attention (Fanelli, 2010).

Lack of Due Consideration

They may know little of it, and they may not give it due consideration. That's the expression, to reject it without due consideration. Some scholars have used that term, or the equivalent (Roberts, 2019).

They may look at it very superficially, perhaps read the abstract, perhaps only skim.

They may attempt to interpret it, but they'll interpret it based on their own biases, their own understanding of what the author intended (Nicholson, 2019).

Poor Writing from Researchers

So they may totally misinterpret what's being said, what the author is trying to convey. Perhaps the author was not a particularly good writer (Pinker, 2014).

Rejection Based on Conflicting Worldviews

The understanding that they take away, a reader takes away, may be a poor one. The reader may object to the paper because it conflicts with his understanding of the field, contradicts the conventional wisdom of his field, doesn't support his biases, so he will reject it (Nickerson, 1998).

Critique of the Author's Conclusion

The reader may be able to look at it, and this is not unbiased, but look at it and say that the data did not support the conclusion, that the author did not honestly look at his own data and reach a sensible conclusion (Greenland & O'Rourke, 2018).

Technical Criticisms of Methodology

The reader may look at it from the viewpoint of statistical interpretation, usage, methodology, research design, and say it's flawed on various technical grounds. They didn't do a good job, they weren't competent, whatever (Gelman & Loken, 2013).

Rejection Based on Journal Reputation

A reader may look at the reputation of the journal and say, oh, it's not worth the papers published on, or the electronic media, as the case may be (Brembs, 2018).

Prejudice Against Researchers

The author may know something of the researchers and say, well, they're a bunch of hacks. I can't trust anything they produce. Their research is garbage (Ioannidis, 2011).

Distrust of Funding Sources

The reader may say, well, I know something of the funding of those guys, and it's not credible because they were funded by big business, and the research was destined to prove the case that big business wants proven, pharmaceuticals, what have you, and I can't trust it on those grounds (Gøtzsche, 2013).

Personal Bias in Interpretation

The reader may read it and assign a very idiosyncratic interpretation to what the author says because of his own biases, his own particular understanding (Bauer, 1994).

The Inescapable Role of Belief in Interpretation

A person can only understand things within the framework of what they currently believe. That's just common sense. How could it be otherwise? (Kuhn, 1962).

The Influence of Personal and Disciplinary Biases

So, what the reader may believe is probably held to be the body of knowledge in their discipline, or their own idiosyncratic interpretation of that body of knowledge, coupled with their biases as human beings (Nickerson, 1998).

The Reality of Scientific Evaluation

So, much research is done. It may not be accepted for various reasons. It may be encountered. It may not be encountered.

Rejecting the Naive View of Scientific Progress

But in any case, to regard it as the simple process outlined at the beginning is wrong. It's really, that's a cartoonish description (Latour, 1987).

---

Barriers to Publication

Not all articles that are submitted to journals get accepted. The process is subject to biases. These determine which studies are published and which are rejected (García et al., 2021).

Publication Bias

The rejection and acceptance of studies contribute to a larger issue of publication bias. Only certain types of research make it into journals (Fanelli, 2010).

Suppression of Studies

Some studies may be suppressed or even retracted in some cases. Retractions can occur for spurious reasons, such as saying things which are deemed to be unacceptable to the powers that be. There are cases where this has been shown to happen (Roberts, 2019).

Bias in Research Funding

Not all proposals are accepted and receive funding due to a variety of factors (Ioannidis, 2005).

Institutional Support and Sponsorship

Researchers may not always have institutional support, as not all institutions are willing or able to sponsor or pay for certain investigators. This complicates the studies that are done (McCook, 2018).

---

Summary

In this essay, I have highlighted the complexities and biases inherent in scientific progress. The naive view that science is simply a matter of publishing and reviewing is deeply flawed. Many factors influence whether research is published, acknowledged, or even encountered, from biases against researchers to technical critiques of methodology. Understanding these challenges reveals the messy, human side of science, far from the clear-cut, rational process it is often assumed to be.

---

References

Bauer, H. H. (1994). Scientific literacy and the myth of the scientific method. University of Illinois Press. https://www.press.uillinois.edu/books/?id=p064364

• Author Note: Harry H. Bauer is a professor of chemistry and science studies.

• Content Note: This book critiques the idea that there is a single, uniform "scientific method," arguing that science is far more complex and varied in practice than is often assumed in public discourse.

Brembs, B. (2018). Prestigious science journals struggle to reach even average reliability. https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2018.00037/full

• Author Note: Björn Brembs is a professor of neurobiology, specializing in behavior and learning in animals.

• Content Note: This paper argues that high-profile scientific journals often fail to maintain the level of rigor required for reliable scientific publishing, suggesting that their impact may distort scientific progress by prioritizing novelty over accuracy.

Fanelli, D. (2010). Do pressures to publish increase scientists' bias? An empirical support from US States Data. PLoS ONE, 5(4), e10271. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0010271

• Author Note: Daniele Fanelli is a researcher focused on the sociology of science and scientific integrity.

• Content Note: This article presents empirical data showing that the pressure to publish frequently correlates with an increase in biased research outcomes, particularly in highly competitive academic environments.

Gelman, A., & Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf

• Author Note: Andrew Gelman is a professor of statistics and political science, and Eric Loken is a psychologist specializing in methodology.

• Content Note: This paper explains how even well-intentioned research can lead to spurious findings due to the inherent flexibility in data analysis and hypothesis testing, a phenomenon they call "the garden of forking paths."

Gøtzsche, P. C. (2013). Deadly medicines and organised crime: How big pharma has corrupted healthcare. Radcliffe Publishing. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4046551/

• Author Note: Peter C. Gøtzsche is a Danish medical researcher and co-founder of the Cochrane Collaboration.

• Content Note: Gøtzsche’s book is a critical exposé of the pharmaceutical industry, accusing it of criminal behavior in its pursuit of profit, to the detriment of public health.

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLOS Medicine, 2(8), e124. https://journals.plos.org/plosmedicine/article/file?type=printable&id=10.1371/journal.pmed.0020124

• Author Note: John P. A. Ioannidis is a professor of medicine and statistics, known for his work on evidence-based medicine.

• Content Note: This landmark article demonstrates why, due to biases, statistical issues, and other factors, a significant proportion of scientific research findings may be incorrect or misleading.

Ioannidis, J. P. A. (2011). An epidemic of false claims. Scientific American, 304(6), 16. https://www.scientificamerican.com/article/an-epidemic-of-false-claims/

• Author Note: John P. A. Ioannidis continues his investigation into the reliability of scientific findings.

• Content Note: In this article, Ioannidis addresses the growing concern about false findings in the scientific literature, drawing attention to the systemic issues that allow unreliable research to proliferate.

Kuhn, T. S. (1962). The structure of scientific revolutions. University of Chicago Press. https://psycnet.apa.org/record/1962-35001-000

• Author Note: Thomas S. Kuhn was a historian and philosopher of science.

• Content Note: Kuhn's book introduced the concept of paradigm shifts in science, arguing that scientific progress often occurs through revolutionary changes in the fundamental frameworks that define a field, rather than through steady, cumulative development.

Landhuis, E. (2016). Scientific literature: Information overload. Nature, 535(7612), 457-458. https://www.nature.com/articles/nj7612-457a

• Author Note: Esther Landhuis is a science journalist.

• Content Note: This article discusses the overwhelming volume of scientific publications, highlighting the difficulties researchers face in keeping up with the growing body of literature.

Latour, B. (1987). Science in action: How to follow scientists and engineers through society. Harvard University Press. https://www.hup.harvard.edu/books/9780674792913

• Author Note: Bruno Latour was a French sociologist and philosopher, known for his work in science and technology studies.

• Content Note: In this book, Latour examines how scientific knowledge is produced and how the social processes involved in scientific work influence its outcomes, providing a sociological perspective on science.

McCook, A. (2018). Many junior scientists need to take a hard look at their job prospects. Science. https://pubmed.ncbi.nlm.nih.gov/29072277/

• Author Note: Alison McCook is a journalist and editor focusing on science and academic integrity.

• Content Note: This article highlights the career challenges faced by early-career scientists, including job insecurity, funding difficulties, and the oversupply of PhDs in certain fields.

Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220. https://psycnet.apa.org/record/2018-70006-003

• Author Note: Raymond S. Nickerson is a psychologist known for his work on cognitive biases.

• Content Note: This paper provides a comprehensive overview of confirmation bias, a cognitive phenomenon where individuals favor information that confirms their pre-existing beliefs and ignore information that contradicts them.

Pinker, S. (2014). The sense of style: The thinking person’s guide to writing in the 21st century. Viking. https://stevenpinker.com/publications/sense-style-thinking-persons-guide-writing-21st-century

• Author Note: Steven Pinker is a cognitive psychologist, linguist, and popular science writer.

• Content Note: Pinker's book is a guide to writing clearly and effectively, grounded in modern cognitive science. It aims to help writers improve their style while taking into account how readers process written information.