Taylor was born in New York, New York. His mother was Jewish and his father was a "WASP" from Tennessee. Taylor played bass guitar in The Gamblers, one of the first rock groups to play instrumental surf music. Its personnel also included Elliot Ingber, a Future member of Frank Zappa's Mothers of Invention, Fraternity of Man and Captain Beefheart's The Magic Band; Bruce Johnston, half of the Bruce and Terry duo with Terry Melcher from 1962–66 and longtime "sixth" member of The Beach Boys, for a time brother Mel Taylor, and guitarist-songwriter-bandleader Derry Weaver, who would record and perform in several capacities during the early 1960s. The Gamblers had a local hit in the Los Angeles area with "Moon Dawg" and Taylor played on the recording.