Risk-sensitive and risk-neutral multiarmed bandits

Risk-sensitive and risk-neutral multiarmed bandits

0.00 Avg rating0 Votes
Article ID: iaor200954093
Country: United States
Volume: 32
Issue: 2
Start Page Number: 374
End Page Number: 394
Publication Date: May 2007
Journal: Mathematics of Operations Research
Authors: , ,
Keywords: stochastic processes, risk
Abstract:

For the multiarmed bandit, the classic result is probabilistic: each state of each bandit (Markov chain with rewards) has an index that is determined by an optimal stopping time for that state's bandit, and expected discounted income is maximized by playing at each epoch a bandit whose current state has the largest index. Our approach is analytic, not probabilistic. It uses pairwise comparison in place of stopping times. A simple recursion assigns to each state of each bandit a utility and an amplification of future utility that depend solely on the data for that state's bandit. These utilities and amplifications determine whether or not one state dominates another. We show that it is optimal to play at each epoch any bandit whose current state is not dominated by the current states of the other bandits. We obtain this result by a coherent analysis that encompasses three models—one with risk–averse exponential utility, one with risk–seeking exponential utility, and one with linear utility and either stopping or discounting. We also show that the risk–seeking case and a model of Nash (1980) are equivalent to each other.

Reviews

Required fields are marked *. Your email address will not be published.