Fraser was born in Southwark, London. Her year of birth was usually attributed as 1933, which she gave when auditioning for her role in I'm All Right Jack, as the Boulting Brothers wanted someone younger for the part. She is actually three years older, confirmed in her autobiography, Liz Fraser ... and Other Characters, published by Signum Books in 2012. Her father was a travelling salesman for a brewery and her mother owned a shop just off the New Kent Road. Family life was disrupted by the Second World War when she was evacuated, initially to Westerham in Kent and then, when that was deemed still too vulnerable to the bombs, to Chudleigh, a village in Devon. Her father died in May 1942, aged 40, when she was 11.