Adaptive value function approximation for continuous‐state stochastic dynamic programming

Adaptive value function approximation for continuous‐state stochastic dynamic programming

0.00 Avg rating0 Votes
Article ID: iaor2013776
Volume: 40
Issue: 4
Start Page Number: 1076
End Page Number: 1084
Publication Date: Apr 2013
Journal: Computers and Operations Research
Authors: , ,
Keywords: programming: dynamic, inventory, neural networks
Abstract:

Approximate dynamic programming (ADP) commonly employs value function approximation to numerically solve complex dynamic programming problems. A statistical perspective of value function approximation employs a design and analysis of computer experiments (DACE) approach, where the ‘computer experiment’ yields points on the value function curve. The DACE approach has been used to numerically solve high‐dimensional, continuous‐state stochastic dynamic programming, and performs two tasks primarily: (1) design of experiments and (2) statistical modeling. The use of design of experiments enables more efficient discretization. However, identifying the appropriate sample size is not straightforward. Furthermore, identifying the appropriate model structure is a well‐known problem in the field of statistics. In this paper, we present a sequential method that can adaptively determine both sample size and model structure. Number‐theoretic methods (NTM) are used to sequentially grow the experimental design because of their ability to fill the design space. Feed‐forward neural networks (NNs) are used for statistical modeling because of their adjustability in structure‐complexity . This adaptive value function approximation (AVFA) method must be automated to enable efficient implementation within ADP. An AVFA algorithm is introduced, that increments the size of the state space training data in each sequential step, and for each sample size a successive model search process is performed to find an optimal NN model. The new algorithm is tested on a nine‐dimensional inventory forecasting problem.

Reviews

Required fields are marked *. Your email address will not be published.