In 1940, Duryea moved to Hollywood to appear in the film version of The Little Foxes. He continued to establish himself with supporting and secondary roles in films such as The Pride of the Yankees and None But the Lonely Heart. As the 1940s progressed, he found his niche as the "sniveling, deliberately taunting" antagonist in a number of film noir subjects (Scarlet Street, The Woman in the Window, Criss Cross, Too Late for Tears, Johnny Stool Pigeon) and westerns such as Along Came Jones and Black Bart, although he was sometimes cast in more sympathetic roles (Black Angel, One Way Street). In 1946, exhibitors voted him the eighth most promising "star of tomorrow".