Let D be a set of positive integers. The distance graph generated by D has all integers ℤ as the vertex set; two vertices are adjacent whenever their absolute difference falls in D. We completely determine the chromatic number for the distance graphs generated by the sets D={2,3,x,y} for all values x and y. The methods we use include the density of sequences with missing differences and the parameter involved in the so called ‘lonely runner conjecture’. Previous results on this problem include: For x and y being prime numbers, this problem was completely solved by Voigt and Walther (Discrete Appl. Math. 51:197–209, 1994); and other results for special integers of x and y were obtained by Kemnitz and Kolberg (1998) and by Voigt and Walther (1991).