Why does Prospero decide to show mercy to his enemies? Do you think Prospero had planned to forgive them from the beginning?

Prospero appears to experience quite a transformation throughout the play. Initially, he's bent on revenge against those responsible for his enforced exile. The arrival of his foes on the island presents him, then, with a golden opportunity to exact vengeance. Even here though there are hints of Prospero's later mercy. He expresses sincere concern for the welfare for those on board ship:

But are they, Ariel, safe? (Act I Scene ii).

At the same time, however, Prospero plays with his enemies like a cat with a bird in its claws, giving them the impression that he will have his revenge, even if it means that innocent men like Gonzalo end up suffering. Prospero wants to keep the shipwrecked men guessing as to what he will do to them. After being imprisoned by Prospero in a grove, Alonso, Antonio, and Sebastian are terrified at what they believe will happen to them, as Ariel points out:

The king, His brother, and yours, abide all three distracted, And the remainder mourning over them, Brimful of sorrow and dismay. (Act V Scene i).

But Prospero has made his point and orders their release. There's a sense that this was indeed his plan all along, as all his actions seem so calculated. Yet his mercy, such as it is, doesn't strike one as being particularly sincere. For one thing, he still openly hates Antonio:

For you, most wicked sir, whom to call brother Would even infect my mouth, I do forgive Thy rankest fault, all of them, and require My dukedom of thee, which perforce, I know, Thou must restore. (Act V Scene i).

But Prospero is not a vindictive man—or not very, at any rate. He wants to be restored to his dukedom, and once he's assured of that, then he has no further need of either magic or vengeance.