Cooperative Markov decision processes: time consistency, greedy players satisfaction, and cooperation maintenance

We deal with multi‐agent Markov decision processes (MDPs) in which cooperation among players is allowed. We find a cooperative payoff distribution procedure (MDP‐CPDP) that distributes in the course of the game the payoff that players would earn in the long run game. We show under which conditions such a MDP‐CPDP fulfills a time consistency property, contents greedy players, and strengthen the coalition cohesiveness throughout the game. Finally we refine the concept of Core for Cooperative MDPs.