Chen was born in Ningbo, Zhejiang Province, Mainland China in 1923. His family was poor and he left school at the age of 12 to serve as an apprentice to a silk merchant. His Father was reportedly a Shanghai-based industrialist, mainly in textiles. When Chen was 22 years old, he was already chief manager of his family Business and owned several shops and factories in Shanghai and Ningbo. In 1949, Chen's family shifted to Hong Kong and set up in Business there. He set up a successful cotton-yarn maker called Nan Fung Mill.