Smokey Robinson wrote "When I'm Gone" and produced it for "The Queen Of Motown" the late Mary Wells as the follow up to "My Guy". After it was recorded in 1964 Mary apparently decided to leave Motown saying that she wasn't receiving her just royalties and the contract she signed was not legal. Mary's contract expired and the single wasn't released for years.

Brenda Holloway had recently signed with Motown and they convinced her to have Smokey Robinson produce "When I'm Gone". The deed was done and in 1965 it peaked at number twelve R&B, number twenty-five on the Billboard Top One Hundred. Both versions had background vocals by The Andantes and the instruments of The Funk Brothers.