This paper considers the relationship between ship-owner and harbour in loading and unloading goods: n ships reach the same harbour at zero time, and also hope to finish loading and unloading goods at the same time. If a ship couldn't finish work before or at its due date, the ship-owner will ask the harbour to pay the penalty; otherwise, the ship-owner will pay prize to the harbour. Together with the normal cost, the harbour needs to consider the best sequence of loading and unloading n ships' goods such that the total cost is minimised. Corresponding to such an NP-hard problem, this paper gives some polynomial time solvable cases, and a branch and bound algorithm with a rapid method for estimating the lower bound for general case.