Born in Brooklyn on June 9, 1931, the same day his father died. His mother Rose (née Sarno), sold olive oil and eventually became a nightclub owner and singer in New York City and Havana. She later married Puerto Rican-born Daniel Santos, and Joe took his name.