Why is Prospero overthrown?

In act 1, scene 2, Prospero explains to Miranda how they ended up on the magical island twelve years ago. Prospero begins by telling Miranda that he was once the powerful Duke of Milan. As Duke of Milan, Prospero became infatuated with studying logic, grammar, geometry, and astronomy, instead of focusing on politics and running the state. While Prospero was "rapt in secret studies," he entrusted his brother, Antonio, to manage his state and affairs. Antonio excelled in his position by granting his political allies powerful positions while Prospero was off studying by himself. Once Antonio established his authority, he managed to sway to the public in his favor and allied himself with the King of Naples, Alonso. With Alonso's assistance, Antonio successfully usurped power and directed his soldiers to take Prospero and Miranda out to sea, where they abandoned them in a small boat. Fortunately, Gonzalo gave them food, necessities, and Prospero's books on magic before they sailed to the island where they currently reside. Essentially, Prospero was overthrown because his ambitious, conniving brother took advantage of his political authority while Prospero was focused on his studies.

Prospero was so engrossed in his studies, in his books and magic, that he neglected his duties as Duke of Milan. He was more than happy to leave the day-to-day administration of the dukedom to his brother, Antonio. This turned out to be a foolish mistake. It's a similar mistake to the one made by King Lear when he divides up his kingdom between his daughters. If someone in a position of authority gives the impression that they're no longer interested in performing their duty, then one can be certain that someone will gladly step into the breach and do it for them.
And that's precisely what happens in relation to Prospero. As Antonio was effectively left in charge of the government, he wondered why he couldn't take over as Duke in form as well as substance. So he made his move, and Prospero was deposed and subsequently banished along with Miranda to a remote island.