Ammons grew up on a tobacco farm near Whiteville, North Carolina, in the southeastern part of the state. He served in the U.S. Navy during World War II, stationed on board the U.S.S. Gunason, a battleship escort. After the war, Ammons attended Wake Forest University, majoring in biology. Graduating in 1949, he served as a principal and Teacher at Hattaras Elementary School later that year and also married Phyllis Plumbo. He received an M.A. in English from the University of California, Berkeley.