Huang was born into a rural intellectual family in Guixi, Jiangxi province in 1904. With the help of local friends and townsmen he was admitted to the elite Whampoa Military Academy. He was loyal, dutiful and brave in battle and caught the attention of Chen Cheng, his superior and senior military instructor who introduced him to Generalissimo Chiang Kai-shek. By the end of 1927, he was promoted to regimental commander of the 9th corps, in 1928 he was reassigned as regimental colonel of the 11th division, which Chen Cheng was the commander. In 1928 he was enrolled in the Army War College and after graduation he was promoted to 32nd brigade commander in the 11th division, now as part of the 18th corps, again Chen Cheng was the Corps superior general officer.