Peruvian born, the son of a professional photographer (young Vargas did the photo retouching). He studied in France and moved to New York in 1916. Vargas was "discovered" by Florenz Ziegfeld as the artist was doing window displays. From 1919, he painted the Ziegfeld girl for twelve years. From the late 1920s, Vargas worked for a number of movie studios, doing everything from stars portraits to movie posters to set design (he was an accomplished draftsman).

Vargas’ career was, in many aspects, one that paralleled the fortunes of fellow illustrator George Petty. When Petty and Esquire parted company over money and content, Vargas was hired. Vargas’ name, signature-wise, was changed to “Varga” by publisher David Smart so Esquire could contractually exercise more control over its pin-up “product” (didn't want another Petty on their hands). Vargas also followed Petty in working for Jantzen, True Magazine, and doing ice skating programs (though Petty followed Vargas in doing movie studio work and Old Gold promotions).

When a number of illustrators moved from Esquire to Playboy in the late 1950s and early 1960s, it wasn’t Petty who went with them (he had returned intermittently to the Esquire fold), but Vargas. Playboy gave Vargas back his full name and many years of respect and consistent employment. Vargas was so loyal during his Playboy years that he turned down virtually all outside opportunities. He painted over 150 Vargas girls for the magazine in sixteen years.

Like many pin-up artists, Vargas was a one-woman man. He eloped with Anna Mae Clift in 1930. Their marriage lasted until her death in 1974.