Baltz was born in New York City, New York. He is best known for his role as Officer Marvin Nash, an LAPD police officer in the famous torture scene of the 1992 Quentin Tarantino film Reservoir Dogs. Baltz also starred in another film written by Tarantino, 1994's Natural Born Killers, as Roger. His other film credits include the 1990 drama film Dances With Wolves, the 1997 action film Face/Off and the 1998 political satire Bulworth. Notably, Baltz starred in Paul Thomas Anderson's directorial debut in the 1993 short film Cigarettes & Coffee.