Piecewise affine approximations for the control of a one-reservoir hydroelectric system

We analyze the computation of optimal and approximately optimal policies for a discrete-time model of a single reservoir whose discharges generate hydroelectric power. Inflows in successive periods are random variables. Revenue from hydroelectric production is represented by a piecewise linear function. We use the special structure of optimal policies, together with piecewise affine approximations of the optimal return functions at each stage of dynamic programming, to decrease the computational effort by an order of magnitude compared with ordinary value iteration. The method is then used to obtain easily computable lower and upper bounds on the value function of an optimal policy, and a policy whose value function is between the bounds.