Gene Williams was born in El Dorado, Kansas on December 22, 1923, the second of two sons to Dr. L. L. and Helen Varner Williams. He and his brother Wade were both talented artists.

Williams enrolled at the University of Kansas in 1941 in pre-med, but his studies were interrupted by World War II. He served as an infantry rifleman and company aid-man in Europe, returning to the United States in the spring of 1946. Before he left KU for the war and while serving as cartoonist for the University Daily Kansan, Jayhawker, and Sour Owl, he sketched the "Fighting Jayhawk," a version of KU's mascot involving a scowl.

Williams returned to KU to study in the fall of 1947, majoring in fine arts, but dropped out soon afterward, after his father died in November. Eventually he moved to Taos, New Mexico, joining his brother's art studio and studying fine arts at the University of New Mexico.

In 1948 Williams married Virginia "Ginny" Tolle. After his brother's death from Hodgin's Diseases, Williams returned to his pre-med studies, eventually receiving his medical degree from KU in 1954.

After completining his internship and residency training in Kansas City-area hospitals, Williams and his wife moved to Phoenix, Arizona where Williams opened a private practice in ear, nose, and throat medicine. The couple had two children, Betsy and Wade.

The family moved back to El Dorado in 1970, and on July 4, 1979 Williams and a friend named John Siefkes went up in Siefkes' hot air balloon. Due to an accident with the balloon, both men died.