We consider an assembly system with multiple stages, multiple items, and multiple customer classes. The system consists of m production facilities, each producing a different item. Items are produced in variable batch sizes, one batch at a time, with exponentially distributed batch production times. Demand from each class takes place continuously over time according to a compound Poisson process. At each decision epoch, we must determine whether or not to produce an item and, should demand from a particular class arise, whether or not to satisfy it from existing inventory, if any is available. We formulate the problem as a Markov decision process and use it to characterize the structure of the optimal policy. In contrast to systems with exogenous and deterministic production lead times, we show that the optimal production policy for each item is a state‐dependent base‐stock policy with the base‐stock level nonincreasing in the inventory level of items that are downstream and nondecreasing in the inventory level of all other items. For inventory allocation, we show that the optimal policy is a multilevel state‐dependent rationing policy with the rationing level for each demand class nonincreasing in the inventory level of all nonend items. We also show how the optimal control problem can be reformulated in terms of echelon inventory and how the essential features of the optimal policy can be reinterpreted in terms of echelon inventory.