Reason: (4th) On Variability, Probability, and the Illusion of Prediction
Why Mathematical Descriptions of Unstable Systems Mislead More Than They Reveal
Summary
Across this discussion, the central theme has been the nature of probability, variability, and the limits of applying mathematical models to the real world. The conversation unfolded through iterative drafts and refinements of appendices, each focusing on a different dimension of the argument.
Probability as Language
A recurring point is that mathematics is a form of language. It is not the world, but a way of describing selected features of the world. Probability, in particular, is a specialized language for describing patterns of variability—patterns often hidden, inferred only through counts, distributions, and frequencies. This makes probability distinct from ordinary language, though it always depends on natural language for interpretation. The asymmetry is clear: everything in mathematics can be translated into words, but not everything in words can be translated into mathematics.
Variability and the Dual Sense of Probability
The world is governed by variability. Probability describes this in two senses. First, as variability itself, which can be captured in ordinary descriptive language. Second, as the formal language of probability, which encodes variability as distributions and ratios. The danger lies in treating the latter as if it were the former—confusing the linguistic device with the feature of the world it attempts to describe.
Situational Nature of Probability and Measurement
Probability is always situational. What counts as an event, what counts as an outcome, and what constitutes a situation are all human decisions. Dice, coins, or card draws are stable situations, where events and outcomes can be cleanly enumerated. But in medicine, psychology, or nutrition, the situation is much less stable. Measurement itself is a decision: what to count, how to count, and which instruments to use. Neither probability nor measurement inheres in the universe in a Platonic sense—they are linguistic and conceptual frameworks we impose.
Variability and the Reification of Probability
There is a strong tendency to reify probability, treating it as if it were a real property of the world. Yet probability does not exist “out there.” Variability exists, but probability is our description of variability. In domains like dice, coins, or ballistics, probability works well and has predictive bite. In open systems like weather forecasting, medicine, or stock markets, it is often reified, creating the illusion of precision where only aggregate tendencies exist.
Probability, Chaos, and the Limits of Description
Both probability and chaos mathematics attempt to describe variability, but neither describes the world in full. Probability works best in closed, enumerable systems. Chaos points to instability and sensitivity to initial conditions but does not yield specific predictions either. Both provide partial, pattern-based descriptions, not comprehensive accounts of reality.
Assumptions and Meta-Assumptions
A central conceptual distinction emerged between assumptions and meta-assumptions. Assumptions are the explicit conditions inside a model (independence, identical distribution, stability). Meta-assumptions lie one level higher: that the model applies to the real world, that the right assumptions have been chosen, and that if those assumptions hold the model will map reality. Mathematics can prove internal coherence, but not applicability. The leap from model to world rests on meta-assumptions, often unacknowledged.
Parallels in Scholarship
Several scholars echo these themes:
· Nancy Cartwright speaks of nomological machines, where laws hold only under structured setups.
· George Box declared all models wrong but some useful, hinting at the gap between simplification and applicability.
· Imre Lakatos described hard cores and protective belts in scientific programmes, showing how assumptions shield models.
· Joel Michell critiqued the untested assumption that psychological constructs are measurable like physical quantities.
· Ian Hacking analyzed different “styles of reasoning,” each with its own standards of evidence.
All recognize the gap between models and the world, though often under different terminologies. The assumption/meta-assumption framework makes the issue clearer.
Concrete Examples
Several illustrative examples were developed:
· Ballistics: Accuracy and precision show how probability can capture variability operationally, but only once the situation (shooter, weapon, environment) is defined.
· Walnut Cracking: Even simple certainty (“the walnut will break”) depends on situational definitions. Change the setup, and outcomes change.
· Medical Prognosis: Stroke risk expressed as “50–50” illustrates how aggregate data is misapplied to individual cases.
· Nutrition Research: Claims that certain foods reduce disease risk by percentages exemplify the reification of probability in unstable, confounded systems.
The Bottom Line
The consistent conclusion is that variability is real, but probability is our language for describing it. In closed, stable systems, probability works well. In open, unstable systems, its applicability rests on meta-assumptions that are often unjustified. Reifying probability leads to misplaced confidence, treating linguistic patterns as though they were properties of the world.
In the end, the dream of prediction remains largely illusory. Probability describes aggregates, not individual outcomes. The world is variable, and perhaps more unstable and less intelligible than we often assume.
References (APA with annotations)
Box, G. E. P. (1976). Science and statistics. Journal of the American Statistical Association, 71(356), 791-799.
Annotation: Box introduced the maxim, "all models are wrong, but some are useful." He emphasizes that models are simplifications, never literal truths. This matches the assumption/meta-assumption distinction: models contain assumptions, but the claim that usefulness implies real-world correspondence is itself a meta-assumption.
Cartwright, N. (1999). The dappled world: A study of the boundaries of science. Cambridge University Press.
Annotation: Cartwright coins the term "nomological machine" to describe structured setups that enforce conditions under which laws appear to hold. This corresponds directly to the distinction: assumptions belong inside the machine; the claim that the law extends beyond the machine is a meta-assumption.
Hacking, I. (1982). Experimentation and scientific realism. Philosophical Topics, 13(1), 71-87.
Annotation: Hacking discusses how different sciences adopt different "styles of reasoning," each with its own standards of validity. The critical leap comes when these styles are treated as tethered to reality. That leap is meta-assumptive.
Lakatos, I. (1978). The methodology of scientific research programmes. Cambridge University Press.
Annotation: Lakatos distinguishes between the "hard core" of a programme and its "protective belt" of auxiliary hypotheses. The assumption/meta-assumption distinction parallels this: assumptions are in the belt, while the meta-assumption is that protecting the belt preserves the reality of the hard core.
Michell, J. (1999). Measurement in psychology: A critical history of a methodological concept. CambridgeUniversity Press.
Annotation: Michell critiques the unexamined assumption that psychological constructs such as intelligence or anxiety are measurable on numerical scales. The meta-assumption is that if numbers can be assigned, they behave like physical measurements. This directly supports the claim that models rest on untested meta-assumptions.
The Complete Series:
1st: https://ephektikoi.substack.com/p/reason-1st-on-variability-probability
2nd: https://ephektikoi.substack.com/p/reason-2nd-on-variability-probability
3rd: https://ephektikoi.substack.com/p/reason-3rd-on-variability-probability
4th: https://ephektikoi.substack.com/p/reason-4th-on-variability-probability
5th: https://ephektikoi.substack.com/p/reason-5th-on-variability-probability