Rufus Payne was born in 1883 or 1884 on the Payne Plantation in Sandy Ridge, Lowndes County, Alabama, to formerly enslaved parents. Around 1890 the family moved to New Orleans, where Payne absorbed jazz and blues off the Melpomene Street corners rather than being taught them; friends, mocking the flask of tea and moonshine he carried, nicknamed him Tee Tot. He returned to Alabama around 1914-15 and built an itinerant act across Montgomery, Lowndes and Butler counties — juke joints one night, white garden parties the next; historian Alice Harp remembered him as "a downright dapper figure," not the scruffy busker later legend made him. His son Henderson, kept from him by a disapproving grandmother, recalled sneaking off to hear him play. Payne never recorded. He died in 1939 in a Montgomery charity hospital and was buried in an unmarked grave; no photograph of him is known to survive.