(2403.12324) Towards a pragmatic information theory

View a PDF of the article Towards a Pragmatic Theory of Information, by Edward D. Weinberger

Show HTML PDF (experimental)

Abstract:Standard information theory is silent on the amount of meaning conveyed by information. We fill this gap with a rigorously justifiable quantitative definition of “pragmatic information” as the amount of meaning in a message that is relevant to a particular decision (as opposed to the value the decision maker assigns to that message). We posit that such a message updates a random “state of the world” variable, $\omega$, that informs the decision. The pragmatic information of a single message is then defined as the Kulbach-Leibler divergence between the prior and actualized probability distributions of $\omega$; the pragmatic information of a set of messages is defined as the expected value of the pragmatic information values ​​of the messages in the set. We then show, first, that the pragmatic information of a single message is the expected difference between the shortest binary encoding of $\omega$ under the {\it a priori} and updated distributions, and, second, that the average of the pragmatic values ​​of individual messages, when sampled a large number of times from the ensemble, approaches its expected value.

Pragmatic information thus defined has many expected properties, such as non-negativity and additivity for independent decisions and “pragmatically independent” messages. We also outline three applications: the first is a single game of a slot machine, aka “one-armed bandit”, with an unknown winning probability; the second is a characterization of the rate of biological evolution in a simple model of reproduction with mutation (the so-called “quasi-species” model); the third is a reformulation of the efficient market hypothesis of the financial economy. We conclude by emphasizing the importance of the receiver’s computational capacity in the processing of pragmatic information.