Mullard was born in a humble background in Islington, London. He started work at the age of 14 as a butcher's assistant, and joined the Army at 18. It was there that he began boxing, becoming champion of his regiment. When he left the army after three years, he had a short stint at boxing professionally. This ended after 20 fights over three years, following a knock-out from which he lost his memory. He married Florence Rose in the second quarter of 1939. He is recorded in the 1939 Register as a general labourer living with his parents at 35 Douglas Street, Islington. He rejoined the army in the Second World War, becoming a sergeant major in the Royal Artillery.