Born in Philadelphia, Harris worked as a nurse, and a bouncer in New York City jazz clubs before he began acting, and served as a medic in the United States Army during World War II. After hanging out with many struggling actors, he took a dare and auditioned for his first role and was cast as the father in Nothing But a Man, a critically acclaimed 1964 film about black life in the South starring Ivan Dixon and Abbey Lincoln.