From the mid-1970s on, Carey was a member of Mel Brooks' comedy troupe, appearing in featured roles in films such as High Anxiety, Silent Movie, and History of the World, Part I. He also appeared in supporting roles in other films and on television, and was seen in scores of commercials, including some T-shirt ads for Hanes, and an ad for Nabisco's snack cracker Better Cheddars where he sang. He is most remembered for his role as the diligent but somewhat obsequious and passive-aggressive Officer Carl Levitt, which he played from 1976 to 1982 on the TV series Barney Miller.