Frank Brown was born 10 October 1915 in Cuberstone, Maryland to James and Mary Ann Brown. After growing up in Annapolis, MD, Frank served for five years in the US Army, fighting in both the European and Pacific theatres of WWII. After returning to America, Brown attended Washburn University and Central Baptist Seminary. He then became Assistant Pastor at Shiloh Baptist in Topeka, Kansas; this was the first time the Baptist State Convention appointed an Assistant Pastor. He then served the churches of Horton and Holton, Kansas before becoming the pastor at Greater Second Baptist Church in Topeka.

Brown also participated in numerous community groups. He served as president of the Lawrence, Kansas branch of the NAACP. He also served on the Mayor's Advisory Council and on the Human Relations Council; he was the first African American to serve on the latter, Lawrence Council. He was president of the Inter-denominational Ministerial Alliance, Baptist Ministerial Alliance of Lawrence and Topeka, and was a liaison officer for the Topeka public schools, working with junior high students as a counselor.

On 5 August 1944, Frank Brown married Fannie Louise Watson. Fannie was born on 18 September 1920 in Topeka, Kansas. She attended Topeka Public Schools and both the University of Kansas and Washburn University; she received a Masters Teaching Certificate in Cosmetology from Walker's College of Beauty in Chicago, Illinois. Fannie owned Sunflower Beauty Salon for 32 years. She also established the barber and beauty department at Kansas Technical Institute where she was a cosmetology instructor for seven years. Fannie was active in the church, heading the Young in Action program. She also worked with various civic organizations including the Red Cross.

Frank and Fannie Brown had one son, Frank, Jr. and three daughters, Melody, LaVenzette, and Yvonne. A drunk driver killed Melody when she was only six years old; in 1974 Fannie Brown founded the Melody Brown Memorial Fund. Frank Brown died 22 June 1997 and Fannie Brown died 16 May 2002.