Why the Sun Will Rise Tomorrow: Hume's Problem of Induction, Grue, and the Limits of Learning

The Hook: A Turkey That Had Learned Too Well

A turkey is born one morning on the farm and gets fed. An attentive animal, a good observer. It records the observation but draws no hasty conclusions—a single data point is worthless. So it keeps collecting. It is fed on warm days and cold, on Wednesdays and Thursdays, in rain and in sun. Every morning it adds a new observation to its dataset: "In the morning, I am fed." Week after week the evidence grows. The sample becomes large, varied, statistically overwhelming. Finally, one morning in late autumn, with the confidence of a well-calibrated model, the turkey draws its inductive conclusion: "I am always fed in the morning." It is the morning of the fourth Thursday in November. It is Thanksgiving.

This parable—usually credited to Bertrand Russell, who told a similar story about a chicken—is no children's joke. It is perhaps the most compact introduction to one of the deepest and most uncomfortable problems in all of philosophy: by what right do we actually infer from the observed to the unobserved, from the past to the future, from the sample to the population? The turkey did exactly what every good scientist, every statistician, and every learning system does: it generalized from reliable, plentiful, varied data. And it was catastrophically wrong—not because of a calculation error, but because the procedure itself, indispensable as it is, carries no built-in guarantee.

For someone like Sven, who builds systems for a living that learn from data, generalize patterns, and make predictions, this question has a particular edge. Every model that infers from training data to unseen cases is, at bottom, this turkey. And as we shall see, that is no mere analogy: the problem of induction, which David Hume formulated in 1748, has re-emerged at the heart of modern machine learning as a mathematical theorem. This article takes you along the full route—from Hume's sobering argument through Nelson Goodman's devious riddle built from an invented word, to the surprising insight that there can be no assumption-free learning, neither in humans nor in machines.

Part 1: Hume's Guillotine

Two Kinds of Knowledge

David Hume, the Scottish philosopher of the Enlightenment, divided all objects of human reason, in his Enquiry Concerning Human Understanding (1748), into two classes. On one side stand the relations of ideas: the truths of mathematics and logic. That the angles of a triangle sum to 180 degrees, or that three times five equals half of thirty, we know with certainty, by thought alone, without stepping out into the world. Their denial leads to contradiction. They are necessarily true, but they say nothing about how the world actually is.

On the other side stand the matters of fact: that the sun will rise tomorrow, that bread nourishes, that fire warms. Their denial is no contradiction. That the sun will not rise tomorrow is perfectly conceivable; we can imagine the case without any error of thought. And precisely for that reason we cannot know matters of fact by pure reasoning. How, then, do we know anything about facts that reach beyond our immediate perception and memory?

Hume's answer: through causation, and all knowledge of cause and effect comes from experience. We never see the causal power itself—we only see that one event regularly follows another. Billiard ball A strikes ball B, and B rolls off. What we observe is nothing but a sequence: first contact, then motion. The "necessary connection" between the two we do not see; we supply it in the mind after having lived through the sequence often enough. Causal knowledge is therefore inductive knowledge through and through: generalization from repeated experience.

The Decisive Question

And now Hume poses the question that sets everything trembling. Every inductive inference—from "bread has always nourished me so far" to "bread will nourish me tomorrow too"—tacitly presupposes a bridge. The bridge reads: the future will resemble the past, or more generally, the cases of which we have no experience resemble those of which we do. The Stanford Encyclopedia of Philosophy calls this connecting link the Uniformity Principle, or the principle of the uniformity of nature. Without it, no inductive inference has any force at all. With it, everything seems to work.

So the real question is: by what do we justify the Uniformity Principle itself?

Hume shows with merciless clarity that there are only two possible routes to justification—and that both are blocked.

The first route would be a demonstrative, that is, deductively logical, justification. But it fails immediately. For, as we saw, the denial of the principle is no contradiction. A world in which the laws of nature flip tomorrow, in which bread suddenly poisons and fire suddenly chills, is imaginable without contradiction. From pure logic, then, the uniformity of nature cannot be derived. Deduction is powerless.

The second route would be an empirical, that is, experience-based, justification. One might say: "But the Uniformity Principle has proven itself! In the past, the future always resembled the past; nature has so far been uniform, so it will continue to be." But here the trap snaps shut. This argument presupposes exactly what it is meant to prove. It infers from past uniformity to future uniformity—and this inference is itself an inductive inference, which in turn requires the Uniformity Principle. We justify induction with an induction. That is a circular argument, a going-round-in-circles (petitio principii).

With that, Hume's argument is complete, and in its structure it is of a striking, almost brutal simplicity. Inductive inferences can be justified neither deductively (that would demand a logical necessity that does not exist) nor inductively (that would be circular). No third route is in sight. So, Hume concludes, there is no rational justification of induction at all. This fork—deductively impossible, inductively circular—is often called Hume's fork or, more drastically, Hume's guillotine.

What Hume Does Not Claim

It is important not to misread Hume's position, for it is often caricatured. Hume does not call for abandoning induction. He was no doomsday skeptic who stopped getting out of bed in the morning because the floor might have vanished. On the contrary: Hume was convinced that we simply cannot do otherwise than reason inductively. Only the reason for this is not rational but psychological. It is custom, or habit. After we have lived through fire and warmth together often enough, the sight of fire automatically produces the expectation of warmth—not because reason commands it, but because our nature is built that way. Custom, Hume says, is "the great guide of human life."

That is the real provocation. Hume does not saw off induction, but its rational foundation. He shows that the most reliable epistemic practice we have—the basis of all natural science—rests on no logical rock, but on a deep, inescapable habit of our minds. We are, one might say, cognitively condemned to resemble the turkey. The only question is whether we are wiser than it.

Part 2: Two Hundred Years Later It Gets Worse—Goodman's New Riddle

A Word That Splits the World

One might think the problem of induction had been definitively formulated with Hume and that the rest was commentary. But in 1954 the American philosopher Nelson Goodman, in his slim and consequential book Fact, Fiction, and Forecast, dropped a second bomb—a problem he called the new riddle of induction, and which in a certain sense is even more unsettling than Hume's old one. For Hume's riddle asks whether induction is justified at all. Goodman's riddle asks something finer and more practical: which inductions are good, and which are bad? And it shows that we cannot answer this question as easily as we thought.

Goodman's tool is a single invented word: grue (a portmanteau of green and blue). The definition is a small, innocuous-looking construction with a time index. An object is grue if it was either examined before some fixed future time t and found green, or is examined after t and is blue. Let us set t, say, to January 1, 2050.

Now consider all the emeralds ever examined. Every single one was green. This observation supports, quite naturally, the hypothesis: "All emeralds are green." Good, healthy inductive inference.

But now comes Goodman's stroke. Every one of those examined emeralds was also grue. For each was examined before 2050 and was green—and that is exactly what satisfies the first half of the grue definition. The very same evidence, the same emeralds, the same observations, therefore support with exactly the same logical warrant the hypothesis: "All emeralds are grue."

And these two hypotheses predict incompatible things about the future. "All emeralds are green" says: an emerald we first dig up in 2051 will be green. "All emeralds are grue" says: the same emerald, because it is examined after 2050, will be blue. The same data, the same inductive logic—and two opposite forecasts.

Why This Is So Disturbing

One is tempted at first to wave the trick away: "grue is obviously an artificial, twisted predicate with a built-in point in time—green is natural, grue is a cheat." This instinct is right, but to justify it is astonishingly hard. Precisely there lies the sharpness of the riddle.

Goodman answers the objection with a symmetry trap. We take grue to be "time-dependent" and green to be "timeless." But that holds only because we take green and blue as our basic terms. Introduce a second coined word—bleen (an object is bleen if it is examined before t and is blue, or examined after t and is green)—and we can turn the whole language inside out. In a language whose basic colors are grue and bleen, green suddenly becomes the time-dependent, suspicious predicate: "green" would then mean "grue if examined before t, otherwise bleen." From the standpoint of grue-speakers, we are the oddballs with the twisted, time-indexed concepts. There is no purely logical, language-neutral criterion that singles out green and condemns grue. The apparent self-evidence with which we project "green" and reject "grue" has no formal foundation.

With that Goodman has shifted the problem from Hume's level to a new one. Hume's problem was the justification of induction as such. Goodman's problem is distinguishing good induction from bad—the problem of projectible predicates. Some predicates (green, solid, electrically conductive) we may project into the future in good conscience; others (grue, bleen, and infinitely many more we could invent) we may not, even though they are just as well confirmed by the evidence so far. Logic alone does not tell us which is which. Any body of data that seems to confirm a generalization confirms, with equal formal warrant, infinitely many incompatible generalizations.

Goodman's Own Answer: Entrenchment

Goodman himself did not stop at the diagnosis but proposed a therapy—one that at first strikes many as an admission of defeat, but which is deep. His key concept is entrenchment. What makes "green" projectible and "grue" not? Not its logic, not its closeness to nature, but its history. "Green" is a predicate that has been used in countless successful predictions of the past; it is deeply entrenched in our linguistic and scientific practice. "Grue" has no such track record; no one has ever forecast successfully with it. Projectible are the predicates that have proven themselves in projection.

One sees at once the kinship with Hume's solution: just as Hume traced induction back to custom rather than logic, Goodman traces the choice of good predicates back to ingrained practice rather than to a formal criterion. Both say, at bottom: the justification lies not in pure reason but in the history of our successful dealings with the world. That is no mathematical guarantee. But it may be the most honest thing that can be said. Let us keep hold of this thought—the entrenchment of successful concepts—for it will meet us again, in unexpected guise, among the learning machines.

Part 3: The Rescue Attempts

A problem of this force has, as one would expect, produced whole libraries of answers. None is regarded as decisively conclusive, but the most important repay a look, because each embodies a different conception of what "justification" is even supposed to mean.

Popper: We Do Not Induce at All

The most radical escape comes from Karl Popper, and it is a kind of judo move: Popper accepts Hume's argument entirely and claims it is no problem at all—because science, in truth, never induces. What looks like induction is, according to Popper, really a different procedure, which he calls falsificationism, or the hypothetico-deductive method.

On this picture, scientists do not confirm their theories by piling up positive instances. No mountain of white swans, however large, proves the statement "All swans are white"—Hume is right, that cannot be done. But a single black swan refutes it, and this refutation is purely deductive, entirely watertight. Science therefore advances not by verification but by conjecture and refutation: one puts forward bold, substantive, risky hypotheses and then tries with all one's might to destroy them. What withstands the assaults counts, provisionally, as corroborated, but never as proven. The logical trick is that the refutation of a universal statement by a single counterexample has the form of the valid inference modus tollens—pure deduction, to which Hume had no objection.

Elegant as this is, it carries a price that many find too high. For if corroboration really means only "not yet refuted," then it gives no reason at all to trust a theory for the future. Why should I fly in a well-tested but logically never-confirmed airplane? Strictly speaking, Popper can give no reason to bet on the best-corroborated theory going forward without secretly arguing inductively again ("it has proven itself, so it will continue to prove itself"). Critics therefore regard Popper's solution as a verbal evasion rather than a genuine dissolution: the practical action problem—what should I bet on tomorrow?—remains exactly where Hume left it.

Reichenbach: The Pragmatic Wager

A quite different, appealingly sober approach comes from Hans Reichenbach. He concedes: Hume is right, we cannot prove that induction works. But, says Reichenbach, we can prove that it is our best possible bet—a pragmatic vindication. The argument runs thus: suppose the world has some stable statistical structure, some reliable limiting frequencies. Then the inductive method is guaranteed to be the procedure that uncovers this structure in the long run; it converges to the true frequencies, if there are any. And suppose the world has no such structure—then no method works at all, and by using induction we have lost nothing.

Reichenbach's famous image is that of the fisherman. I do not know whether there are any fish in the lake. But I do know: if there are, I will catch them only if I cast my net; if I do not cast it, I certainly catch nothing. So I cast—not because I know it will work, but because it is the only action that leads to success in the favorable case and worsens nothing in the unfavorable one. Bertrand Russell drove the stakes to their peak: if there were no answer to Hume's problem, he wrote, "there is no intellectual difference between sanity and insanity." Reichenbach's wager saves reason not by proof, but by the insight that under uncertainty it is the dominant strategy.

The catch: Reichenbach shows only that some convergence procedure is rational, not that our particular induction is the right one. And here Goodman waves from the background: the grue-induction converges too—only toward a different "true" frequency. The pragmatic wager tells us that we should cast the net, but not which net.

Bayesianism: Degrees of Belief

The approach most influential today in science and AI gives up the search for certainty altogether and replaces it with the bookkeeping of probability. Bayesianism treats beliefs not as true-or-false, but as degrees of credence between 0 and 1. One starts with a prior probability (the background knowledge before the data) and updates it with each new datum, by Bayes's theorem, to a posterior probability. Every white swan raises the probability of the hypothesis "all swans white" a little; it does not prove it, but it shifts belief in its direction.

That is mathematically clean and practically enormously powerful—it is the backbone of modern statistics and of large parts of machine learning. But Hume is not thereby refuted, only elegantly relocated. For two questions remain. First: where does the prior distribution come from? It is itself an assumption that cannot be won from the data—and precisely here Goodman's grue strikes again, for the grue-fan simply starts with a different prior that favors grue-hypotheses, and his Bayesian arithmetic is just as consistent as ours. Second, the whole machinery presupposes that the rule "update by Bayes and project the raised probability into the future" is reliable—and that is again an inductive assumption. Bayesianism does not justify induction from the outside; it merely gives it a precise, quantitative language. What looks like a solution is in truth a high-resolution description of the problem.

Part 4: The Surprising Return—Induction in the Machine

Every Model Is a Turkey

Here the story becomes concrete for Sven. For what Hume and Goodman circled with emeralds and sunrises is the exact activity of every supervised learning procedure. A model is given training examples—pairs of input and correct output—and is to learn from them a function that yields the right output even for unseen inputs. The ability to infer from the training data to new cases is called, in practice, generalization. And generalization is nothing but induction cast in mathematics: the leap from the observed (training set) to the unobserved (test set, production).

With that, machine learning inherits Hume's problem in full force. Why should a function that fits the training data well also fit future data well? Only if the future resembles the past—if the test data come from the same distribution as the training data. Precisely this assumption (in the jargon: i.i.d., independent and identically distributed) is the Uniformity Principle in statistical dress. It is unprovable, and it is constantly violated when the world changes—a phenomenon practitioners know and dread as distribution shift. The turkey suffered a classic distribution shift: up to the fourth Thursday in November one distribution held, afterward another. Its model was not mistrained; the world failed to honor the i.i.d. assumption.

The No Free Lunch Theorem

Now comes the moment when the two-thousand-year philosophical debate tips over into a hard mathematical result. In 1996 David Wolpert proved, for supervised learning (and in 1997, together with William Macready, for optimization), what is today called the No Free Lunch theorem. In a formulation unusually philosophical for this field, it states: averaged over all logically possible problems, all learning algorithms are exactly equally good. No procedure is inherently better than any other; every advantage an algorithm has on one class of problems is exactly offset by an equally large disadvantage on another class. Over the totality of all conceivable worlds, the most sophisticated deep-learning approach performs, on average, exactly as well as pure guessing.

One should pause a moment to see what this is: it is Hume's guillotine, only this time proven as a theorem, not merely argued philosophically. If one truly makes no assumption whatsoever about the structure of the world—if one holds all possible mappings of inputs to outputs equally likely—then no learning can take place, because the training data then say logically nothing about the unseen cases. That was exactly Hume's point: without the presupposition of uniformity, experience does not carry into the future. The No Free Lunch theorem is the formal, machine-readable version of this insight. And Goodman is right there in it too: the "grue-like" mappings, in which the pattern reverses after a certain point in time, are, among all logically possible functions, just as numerous as the well-behaved, stable "green" ones—averaged over all of them, they cancel out.

Inductive Bias: The Indispensable Presupposition

How, then, can machine learning work at all—and it does work, spectacularly? The answer is the flip side of the theorem, and at the same time the resolution of the apparent paradox. Learning works only because no real procedure averages "over all possible problems." Every usable algorithm brings a set of prior assumptions that favor certain hypotheses from the outset and exclude others. These built-in preferences are called inductive bias. As a much-cited 2023 analysis (Kolmogorov complexity and the role of inductive biases, arXiv:2304.05366) holds, and as Tom Mitchell already argued in 1980: assumption-free learning is impossible. Every learning system must possess an inductive bias, or it learns nothing.

And this is no embarrassing weakness to be optimized away—it is the condition of the possibility of learning at all. The inductive bias is the assumption that breaks the No Free Lunch verdict by restricting the set of hypotheses taken seriously. Concretely, this bias is everywhere: a convolutional network (CNN) assumes that local neighborhood and translation invariance matter in images—an object stays the same wherever in the image it appears. A transformer assumes that relations between elements of a sequence can be captured through attention. Almost all procedures carry a deep leaning toward simplicity (a practical embodiment of Occam's razor): among the functions that fit the data, they prefer the smoother, shorter, more simply describable ones—exactly the predicates Goodman would call "green" and not "grue." A model that prefers simplicity tacitly projects the stable hypothesis "always green" instead of the time-twisted "grue."

With that the circle closes in a startling way. Goodman's solution was entrenchment—projectible are the predicates that have entrenched themselves in successful practice. The inductive bias of an AI model is exactly that: an entrenchment, built into the system, of certain hypothesis classes, which blanks out from the start the infinite space of logically equal grue-alternatives. Where Hume set custom and Goodman set ingrained practice, the engineers set architecture and regularizer. It is the same solution in three languages: since logic alone cannot separate the good generalizations from the bad, an extra-logical presupposition must make the choice—for us, nature; for the model, its design.

A Map of the Answers

For orientation, an overview of who treats the problem how, and what counts as "justification" in each case.

The table makes the common pattern visible. From Hume's custom to the inductive bias of modern networks, the answer to the problem of induction is always the same at its core: the gap that pure logic leaves open is closed by a presupposition that is itself not proven but chosen (or inherited, or built in). There is no free lunch—not for the turkey, not for the human, not for the machine.

The Central Takeaway

The problem of induction is no curiosity for seminar rooms, but perhaps the most practical philosophical insight there is for anyone who works with data and learning systems. Its message is twofold, and both halves are useful.

The sobering half: there is no justification of learning from pure logic. No model, no human, no science can derive the future from the past without inserting an unprovable assumption about the uniformity of the world. Anyone who promises that a procedure works "with no assumptions at all," "purely data-driven," "assumption-free," has the No Free Lunch theorem against them—and is mistaken. Every generalization is a wager, backed not by proof but by a presupposition one had better make explicit than hide.

The liberating half: precisely this unavoidable presupposition—the inductive bias—is no blemish, but the tool one can and should work with. Since one must make assumptions anyway, the decisive engineering question is not "How do I learn without assumptions?" but "Which assumptions fit this problem?" The art of machine learning is, to a large extent, the art of choosing the inductive bias—architecture, regularization, prior knowledge, data representation—so that it matches the actual structure of the domain. A CNN wins on images not because it is "better," but because its built-in assumptions about local structure happen to fit images. On another class of problems it would lose exactly as much as it gains here.

The concrete call to action, therefore, is: make your assumptions visible. When a model fails in production, that is almost never a fault of the mathematics—it is a grue-moment, a distribution shift, a case in which the world's tacitly projected uniformity no longer held. Whoever can name their own inductive presuppositions ("I assume users will behave next year as they did last; I assume the relevant patterns are local and simple") can deliberately check when they break, and build guardrails against it. The turkey had no such list. We can have one. That is the whole difference between naive and enlightened induction—not that we avoid the wager, but that we know what we are betting on.

A Closing Question for Reflection

If every act of learning needs an inductive bias, and there is no "neutral," assumption-free standpoint from which one could objectively determine the right bias—where, then, does the extraordinarily successful bias you yourself bring actually come from? Is your deep feeling that "green" is natural and "grue" absurd, that the world is simple and uniform, a discovery about reality—or a presupposition built into you by evolution and culture that has simply proven itself because you live in a world where it happens to fit? And if the latter: would you be willing to revise it, if one day your data stubbornly told you something grue-like—or would you, as befits a good Bayesian, weight your prior so strongly that no evidence could ever overturn it?

Cross-References in the Vault

This riddle draws threads to several other investigations in the vault:

• How Big Is Big Enough? Scaling Laws, Chinchilla, and the Measurement of AI – inductive bias writ large: scaling laws are themselves an inductive wager on the uniformity of the relation between size and performance.
• The Predictive Brain: Predictive Processing and the Illusion of Perception – the brain as an inductive prediction machine, constantly inferring perception from the past.
• Three Pages Against 2,000 Years: The Gettier Problem and What Knowledge Really Is – the kindred epistemological question of when a true belief counts as knowledge.
• The Peak of Not Knowing: Why Psychology's Most Famous Effect May Be Just a Statistical Mirage – how a statistical artifact becomes an apparent discovery, akin to the warning against hasty generalization.
• The Half-Second Before the Will: Libet's Experiment and the Neuroscience of Free Will – also a story about how method and interpretation can produce results that are not actually in the data.
• The Two Boxes: Newcomb's Paradox and the Battle Over Rational Choice – rationality under prediction and the limits of reasoning about future events.

Sources

• Hume, D. (1748): An Enquiry Concerning Human Understanding. Sections IV–V (Sceptical Doubts / Sceptical Solution). Text edition: https://davidhume.org/texts/e/
• Henderson, L.: The Problem of Induction. Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/induction-problem/
• Goodman, N. (1954/1983): Fact, Fiction, and Forecast. Harvard University Press. Overview: Cohnitz, D., Rossberg, M.: Nelson Goodman. Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/goodman/
• Wolpert, D. H., Macready, W. G. (1997): No Free Lunch Theorems for Optimization. IEEE Transactions on Evolutionary Computation 1(1), 67–82. https://ieeexplore.ieee.org/document/585893
• Goldblum, M. et al. (2023): The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning. arXiv:2304.05366. https://arxiv.org/abs/2304.05366
• Thornton, S.: Karl Popper. Stanford Encyclopedia of Philosophy (on falsificationism and induction). https://plato.stanford.edu/entries/popper/
• Hans Reichenbach. Stanford Encyclopedia of Philosophy (pragmatic vindication of induction). https://plato.stanford.edu/entries/reichenbach/

Note: The problem of induction is an open philosophical question; none of the attempted solutions presented is regarded as decisively conclusive. The account follows the scientifically established state of the debate where such exists, and marks contested interpretations as such.

← All articles