Why does Smith think that the government should stay out of economic affairs?

In looking at this question, it's important to understand who was in charge of the government at that particular time. In Adam Smith's Britain, it was mainly the landed aristocracy in charge. In relation to the economy, it was generally thought vulgar for gentlemen to get involved directly in trade or commerce; land was the source of wealth, prestige, and status for the aristocrat. That being the case, Smith doesn't believe that you can really entrust such people with the running of the economy; they simply don't understand how it works. If they interfere in what should be the natural operations of the market, then they are likely to do more harm than good. Economic policy should not be devised by well-meaning amateurs.
The spontaneous, self-interested decisions of actors in the market place—businessmen, consumers, farmers, and traders—eventually conduce to the common good, to growing economic prosperity. This is a completely unintended consequence; it's just what happens when people are allowed to go about their business without the government becoming too deeply involved. The law of unintended consequences also applies to government intervention in the economy. However well-meaning, however beneficial a specific policy may be, it can lead to harmful consequences further down the line. Government shouldn't stay out of economic affairs completely; it still has an important regulatory role to play. But that role is very broad indeed. What the government should do, according to Smith, is to establish general rules that allow the maximum degree of economic freedom, then stand back to allow the invisible hand of the market economy to do its thing.