Cultivation of Lychee began in the region of southern China, Malaysia, and northern Vietnam. Wild trees still grow in rainforest in Guangdong province and on Hainan Island. Unofficial records in China refer to Lychee as far back as 2000 BCE.

In the 1st century, fresh lychees were in such demand at the Imperial Court, that a special courier service with fast horses would bring the fresh fruit from Canton. There was great demand for lychee in the Sung Dynasty (960-1279ACE), according to Ts'ai Hsiang, in his Li chi pu (Treatise on Lychees). It was also the favourite fruit of Emperor Li Longji (Xuanzong)'s favoured concubine Yang Yuhuan (Yang Guifei). The emperor had the fruit delivered at great expense to the capital. In the Chinese classical work, Shanglin Fu, it is related that the alternate name, meaning leaving its branches, is so-called because once the fruit is picked it deteriorates quickly.

The lychee was first described in the West by Pierre Sonnerat (1748–1814) on a return from his travel to China and Southeast Asia. It was then introduced to the Réunion Island in 1764 by Joseph-François Charpentier de Cossigny de Palma. It was later introduced to Madagascar which has become a major producer.