What stood in the way of Napoleon's plan of rebuilding France's empire in North America?

Napolean never, at least publicly, expressed interest in rebuilding the lost French colonies it once had in North America. Instead, he was more focused on building and maintaining a European empire. To attempt to retake its former colony in Quebec would have proved too expensive and risky to do while trying to gain supremacy across the Atlantic.
After the defeat of the French fleet at the Battle of the Nile in 1798, it would have been very difficult, if not impossible, for France to fight a war across the ocean. The British navy was simply too strong for the fragmented and weak French navy.
After Napoleon won control of Spain, the Louisiana territory came under the control of France. Napolean viewed this territory as too big and too unimportant to his interests to bother delegating resources to defend. By selling the territory to the United States, Napolean eliminated the need to defend a large territory in North America from the British. Instead, he could focus on fighting them closer to home in Europe.
Also, by the beginning of the nineteenth century, France was already struggling to hold on to the few territories in the hemisphere that it already had. The Haitian Revolution was a humiliating defeat for France that cost the nation one of its most profitable territories and destroyed an entire army. Napoleon decided it was better to cut his losses in the region, at least until he could secure an unquestioned foothold in Europe.