Cherryvale was founded on the land of the Osage Indians who were pushed out by veterans of the American Civil War looking for land. The first white man to purchase property and settle here was Mr. Abe Eaton, who later sold it to the Kansas City, Lawrence & Southern Kansas Railroad which then became the Leavenworth, Lawrence & Galveston Railway, and eventually came under the control of the Atchison, Topeka and Santa Fe Railway, which operated the line for much of the 20th century. The town site was platted by the railroad in 1871. It was named Cherryvale from its position in the valley of Cherry Creek.
Between 1871 and 1873, at least eleven people vanished in the vicinity of an inn and general store operated by the Bender family about eight miles (13 km) northeast of Cherryvale. Following the disappareance of the Benders in 1873, it was discovered that they had apparently murdered a number of travellers. Between eight and eleven bodies were discovered buried on the premises. Governor Thomas A. Osborn offered a reward of $2,000 for the apprehension of the Benders, but they were never found.
The main part of the town was destroyed by fire in 1873, but was soon rebuilt, and in 1880 Cherryvale became a city of the third and afterwards of the second class. Natural gas, which is used as a factory fuel and for street and domestic lighting, was found here in 1889, and oil several years later.
The city is currently the headquarters for the South Kansas and Oklahoma Railroad, a shortline that runs 511 miles of track in Kansas and Oklahoma. The railroad, owned by WATCO, painted locomotive EMD SD40-2 number 4158, in the high school's colors of blue and white in 2011, and the unit is stationed at the intermodal switchyard , completed 2013, north of Cherryvale.