When his mother died during the flu epidemic of 1919, his father placed Victor and his younger sister, Rosemary, in a children's shelter, and returned to his homeland to seek another wife. His father returned in 1922, with his new wife Lovi Shee, once again forming a household with his two children. During his acting career, Victor was given billing under a variety of names, including Sen Yung, Sen Young, Victor Sen Young, and Victor Young.