Horace Jeremiah Smith was born January 27, 1838 in Susquehanna County, Pennslyvania, the youngest of five surviving children born to Horace and Marilla Smith. In 1856 Horace moved to Oregon, Ogle County, Illinois, entering the Mt. Morris Seminary. While living in Illinois, he served four years as deputy circuit clerk to his brother Mortimer W. Smith and was then elected in 1860 as county treasurer.

Smith started service in the Civil War as first lieutenant of Company K in the 92nd Illinois Infantry. He was made captain of Company B within 6 months of service. He served the remainder of the war in a variety of capacities before mustering out in 1865 in Chicago.

In 1866 Smith moved to Ottawa, Franklin County, Kansas, then only newly opened to Euro-American settlement after the Ottawa tribe sold part of its land to found Ottawa University. Smith opened the first hardware store in Franklin County and operated it for four years.

In 1872 Smith helped establish the Ottawa Savings Bank. He served as its cashier for several years and then bought a controlling interest in the First National Bank, taking charge there as cashier until 1884 when he became bank president. He remained the bank's president until retiring in 1902.

Smith served in several political offices, including as a member of the Kansas House of Representatives, mayor of Ottawa and on its city council, and as a sate senator. He also held positions in several fraternal and service organizations, including the Grand Army of the Republic, the Masons, Knights Templar, and the Elks.

Smith married Mary F. Ward in 1867; Mary had come to Ottawa in 1865 for a teaching contract. Together they had five children who survived to adulthood: Minnie, Nellie, Grace, Jay Ward, and Horace Jean.

Horace J. Smith passed away December 2, 1906.