James grew up in Maysville, Ark., and moved to Los Angeles, where he was married to Nelvia Demma Buckner (ca1906-1938) in 1924. He had blue eyes and brown hair according to his World War I draft card.

The 1940 census finds them living north of Dodger Stadium, at the end of Riverdale Avenue next to the Los Angeles River. He was a roofer for a construction company. Son Thomas Clay (1925-2004) was 15.