Born in New Orleans, Louisiana, Dorsey was a childhood friend of Fats Domino before moving to Portland, Oregon when he was ten years old. He served in the United States Navy in World War II and then began a career in prizefighting. Boxing as a lightweight in Portland in the early 1950s, he fought under the name Kid Chocolate and was reasonably successful. He retired from boxing in 1955 and returned to New Orleans, where he opened an auto repair business as well as singing in clubs at night.
His first recording was “Rock Pretty Baby/Lonely Evening” on Cosimo Mattasa’s Rex label, in 1958. This was followed by the Allen Toussaint-produced “Lottie Mo/Lover of Love”, for the small Valiant label in late 1960 (picked up by ABC Paramount in 1961). These efforts were unsuccessful, but around 1960 he was discovered by A&R man Marshall Sehorn, who secured him a contract with Fury Records, owned by Bobby Robinson. After meeting songwriter and record producer Allen Toussaint at a party, he recorded “Ya Ya”, a song inspired by a group of children chanting nursery rhymes. It went to number seven on the Billboard Hot 100 in 1961, sold over one million copies, and was awarded a gold disc. Although the follow-up “Do-Re-Mi” also made the charts, later releases on Fury were not successful. Dorsey returned to running his repair business, but also released singles on the Smash and Constellation labels in 1963 and 1964.
He was then approached again by Toussaint, and recorded Toussaint’s song “Ride Your Pony” for the Amy label, a subsidiary of Bell Records. The song reached no.7 on the R&B chart in late 1965, and he followed it up with “Get Out of My Life, Woman”, “Working in the Coal Mine” – his biggest pop hit – and “Holy Cow”, all of which made the pop charts in both the US and the UK. Dorsey toured internationally, and also recorded an album with Toussaint, The New Lee Dorsey in 1966. In 1970 Dorsey and Toussaint collaborated on the album Yes We Can; the title song was Dorsey’s last entry in the US singles chart. It was later a hit for the Pointer Sisters under the title, “Yes We Can Can”. With declining sales, Dorsey then returned to his auto repair business.
In 1976 Dorsey appeared on the album I Don’t Want to Go Home by Southside Johnny and the Asbury Jukes, which led to more recordings on his own with ABC Records, including the album Night People. In 1980, he opened for English punk band The Clash on their US concert tour, and also toured in support of James Brown and Jerry Lee Lewis.
Dorsey contracted emphysema and died on December 1, 1986, in New Orleans, at the age of 61.