Keeping punters log-happy: Some properties of a "pristine parimutuel" market clearing mechanism

Is there such a thing as a perfect probabilistic paradise for punters? A place where participants' prior probabilistic prophecies are pulled apart and placed into a philosophically pristine parimutuel posterior, without painful departure from previously perceived pseudo-optimal particulars? I believe the platonic possibility is promising (though to provoke, in practice we puritans may be prevented from partaking in this powerful portal whilst we are paralyzed by the paranoid parlance of the present day).

                                               A pristine market

Well now that I'm out of spit, here are some characteristics of a market for mutually exclusive, exhaustive outcomes that I would like to see:

1. Every participant is forced to invest in a manner that optimizes their long run wealth. They will allocate their wealth to maximize $E^P[\log {\rm wealth}]$ with respect to their best estimate of real world probability $P$.
2. Every participant's estimate of probability, on which 1. is evidently based, must include both private information (from which they might derive a probability measure $R$ say) and the market probabilities themselves (call them $Q$). In contrast, to simply equate $P$ with $R$ in 1. is a dreadful, though rather common, mistake.
3. Criteria 2. is achieved without the need for participants to monitor and respond to price feedback. In contrast the usual mechanism for including market information, as we find in racetrack parimutuels, is to provide provisional estimates of $Q$ in realtime so that punters can react.

The last requirement motivated my ponderings on this topic. In our perceived utopia participants are given inventive to provide their own private probabilities $R$ well before "race time" (as it were). They can then put their feet up and relax safe in the knowledge that 1. will be satisfied automatically.

Such is non-trivial because even investors whose bets are tiny with respect to the overall market volume (and thus have negligible price impact) must quite rationally react to partial price discovery even though in "theory", they should simply bet proportionately irrespective of the odds on offer (a lovely accident mentioned below). I use scare quotes because unless the odds are truly known, $Q$ should really enter the optimization via the back door: an update to $P$ acknowledging the market's ability to aggregate information.

In this setup the market clearing mechanism does the heavy lifting normally performed iteratively and imperfectly. So long as there is a rule governing the manner in which "$R + Q = P$" (figuratively speaking) we can anticipate every player's estimate of $P$ given $Q$, and thereby solve simultaneously for Q and log-optimal allocations that presumably influence Q. Here we consider the simplest rule of that sort. We shall assume P is a convex combination of $R$ and $Q$ with fixed coefficients.

Also for simplicity we consider the case where the market is not subsidized (though that might be an interesting direction for generalization). Then linear pricing forces us to adopt the most straightforward parimutuel clearing mechanism once we known the allocations: divide all the money wagered amongst those choosing the winning horse, in proportion to their bet.

                                       Necessary optimality conditions

Suppose market participant $k$ allocates all his wealth $W^k$ across $I$ mutually exclusive outcomes. Suppose his estimate of probability for the $i$'th state is given by $$\begin{equation} p^k_i = \eta^k r^k_i + (1- \eta^k) q_i \end{equation}$$ where $r^k$ is his best estimate using only private information, and $q_i$ is the market implied probability arrived at by means to be discussed.

As noted this equation is a statement of a seemingly rational philosophy, independent of how the market operates. Investor $k$ might have noticed in the past that his private information adds some explanatory power to the market, but he probably shouldn't ignore the market prices altogether in arriving at the best estimate of real world probability.

We shall further suppose, in what follows, that all $K$ participants are rational in another sense. They wish to optimize the log of their posterior wealth. Now it is well appreciated that if $p^k_i$ are considered fixed this log-optimality leads simply to proportional betting, but that is not the case here. Only $r^k_i$ and $\eta^k$'s are fixed, and we shall attempt to construct allocations $u = \{u^k\}$ and clearing prices $q_i$ that overtly depend on the investments made by participants.

To that end let $u^k$ denote the fraction of wealth investor $k$ invests in the $i$'th state. Suppose that the market clears in parimutuel fashion, meaning that all participants receive the same price. The market probability for the $i$'th state must be $$\begin{equation} q_i = \frac{ \sum_k u^k_i W^k } { \sum_{k=1}^K W^k } = \sum_k u^k_i W^k \end{equation}$$ since we might as well suppose w.l.o.g. that $\sum_k W^k = 1$.

The question then arises, is there a choice of $\{u^k_i \}_{i,k}$ such that investment by each participant is log-optimal? Intuitively one would expect a market equilibrium provided $\eta^k$'s are strictly between $0$ and $1$.

The utility function for the $k$'th investor is $$\begin{eqnarray} U^k( u^k ) & = & E^p\left[ \log\left( \frac{u^k_i}{q_i} \right) \right] \\ & = & \sum_{i=1}^I p^k_i \log\left( \frac{u^k_i}{ \sum_k u^k_i W^k } \right) \\ & = & \sum_{i=1}^I ( \eta^k r^k_i + (1- \eta^k) q_i ) \log\left( \frac{u^k_i}{ \sum_k u^k_i W^k } \right) \end{eqnarray}$$ and by definition of $u^k_i$ the constraints are $\sum_i u^k_i = 1$. Or for every $k$ we might write $g(u^k)=0$ where $g(u^k) = \sum_i u^k_i - 1$. This sets up the first order Lagrangian equations for $\Lambda(u,\lambda) = U(u) - \lambda \cdot g( u )$ where we collect the components $k=1,..K$. As usual for these problems we set $\nabla \Lambda = 0$ because for optimality the derivative of the objective function must be proportional to the derivative of the constraint function. This leads to equations of the form $$\begin{eqnarray} 0 & = & p^k_i( q_i(u)) \left( \frac{1}{u^k_i} - \frac{W^k}{q_i(u)} \right) - \eta^k W^k \log \left( \frac{u_i}{q_i(u)} \right) - \lambda^k, \ \ \ {\rm and} \\ 0 & = & \sum_i u^k_i - 1 \end{eqnarray}$$ relating the allocations $u^k_i$. Notice there are $KI+K$ equations and $KI+K$ free variables, including both the $u^k_i$'s and the $\lambda^k$ multipliers (the $W^k$ are fixed parameters and we have $q_i(u) = \sum_k u^k_i W^k$ as noted above). The solution to this system of non-linear equations defines a pristine parimutuel clearing mechanism.

                                  Comparison to proportional betting

In contrast if we imagine that $q_i$ are not a function of allocations $u^k_i$ but fixed, and further suppose $\eta^k = 1$ then we return to the overly stylized world where participants don't take market prices into account, preferring to believe their own homework is a sufficient statistic. The optimality conditions are instead: $$\begin{eqnarray} 0 & = & \frac{p^r_i}{u^k_i} - \lambda^k, \ \ \ {\rm and} \\ 0 & = & \sum_i u^k_i - 1 \end{eqnarray}$$ It is apparent from the first equation that $u^k_i \propto r^k_i$ and then, from the second, that $u^k_i = r^k_i$. We recover the remarkable accident referred to as proportional betting, where the price $q_i$ does not enter the picture. This works perfectly well for blackjack where $r^k = p^k$ but not most real world games where markets inevitably supplement one's private information.


                                         
                                                     Critique

While I have caged this discussion in market language, it should be apparent that we have derived an algorithm for combining multiple probabilistic forecasts into a single probability distribution $Q = \{q_i\}$, for which there is a large literature. See the references to Ranjan and Gneiting for example.

I shall concede that what is philosophically inconsistent is my use of simple linear pooling to arrive at the individual's subjective probability estimates $p^k = \{p^k_i\}_{i=1}^I$ based on their prior $r^k$ and the final market price $q_i$, given that we then derive a more sophisticated combination scheme for meshing between individuals. Why not use the "better" scheme to combine $Q$ and $R$?

I suppose a flippant response is "why not put a picture of a boy holding a cereal box on the cereal box?". Let me think about a more satisfying answer and get back to you.