Born in New York City, he was the son of Edith Adelson Lerner and Joseph Jay Lerner, whose brother, Samuel Alexander Lerner, was founder and owner of the Lerner Stores, a chain of dress shops. One of Lerner's cousins was the radio Comedian and television game show panelist Henry Morgan. Lerner was educated at Bedales School in England, The Choate School (now Choate Rosemary Hall) in Wallingford, Connecticut, (where he wrote "The Choate Marching Song") and Harvard. He attended both Camp Androscoggin and Camp Greylock. At both Choate and Harvard, Lerner was a classmate of John F. Kennedy; at Choate they had worked together on the yearbook staff. Like Cole Porter at Yale and Richard Rodgers at Columbia, his career in musical theater began with his collegiate contributions, in Lerner's case to the annual Harvard Hasty Pudding musicals. During the summers of 1936 and 1937, Lerner studied music composition at Juilliard. While attending Harvard, he lost his sight in his left eye due to an accident in the boxing ring. In 1957, Lerner and Leonard Bernstein, another of Lerner's college classmates, collaborated on "Lonely Men of Harvard," a tongue-in-cheek salute to their alma mater.