This park is named in honor of Edward J. Baker, St. Charles' great philanthropist who died in 1959. Baker built several buildings that both beautify and serve the community.

During the earliest days of the settlement, this area served as part of Evan Shelby's claim. Shelby's home stood near the present location of the park. The first white settler to come to the area in 1833, Shelby wanted someone to lay out a town on the east side of the river and he found two men who were willing to do just that. Read and Ira Minard acquired Shelby's land in 1835 in exchange for building a dam, mill, and opening a store. Bela T. Hunt bought land from Shelby just prior to Shelby's death in 1837.

By 1838, the area was known as Penny Brickyard. John Penny established this brickyard at the north end of what is now known as Baker Memorial Park. Area homes, such as the Dunhum-Hunt house on Cedar Avenue, were built of bricks from this brickyard. Construction crews found old bricks and kilns beneath the surface while working on street improvements in the late 1950s.