Why is Holden adjusting poorly and withdrawing from social interaction in The Catcher in the Rye?

In The Catcher in the Rye, Holden admits on page one that he's telling the story from some type of mental hospital after "this madman stuff that happened to me around last Christmas just before I got pretty run-down and had to come out here and take it easy." The entire novel is about Holden's withdrawal from society.
Early in the novel, it seems like Holden is just a slacker. The first real hint that something is mentally and emotionally wrong with Holden is when he discusses Allie's baseball mitt and his brother's death:

I was only thirteen and they were going to have me psychoanalyzed and all, because I broke all the windows in the garage. . . I slept in the garage the night he died, and I broke all the goddam windows with my fist, just for the hell of it. . . It was a very stupid thing to do, I'll admit, but I hardly didn't even know I was doing it, and you didn't know Allie."

Another instance in which it is clear Holden is removing himself from society is his desire to run away from New York. Before he expresses his desire run away with Sally, he lectures her as to what he sees as the "phony" aspects of the world, saying,

I hate living in New York and all. Taxicabs, and Madison Avenue buses, with the drivers and all always yelling at you to get out at the rear door, and being introduced to phony guys that call the Lunts angels, and going up and down in elevators when you just want to go outside, and guys fitting your pants all the time at Brooks.

Finally, Holden's desire to remove himself from society relates back to his brother's death and his belief that childhood should be kept intact throughout life. When at the Museum of Natural History, he explains that children, like his sister Phoebe, shouldn't change.

I kept thinking about old Phoebe going to that museum on Saturdays the way I used to. It didn't exactly depress me to think about it, but it didn't make me feel gay as hell, either. Certain things should stay the way they are. You ought to be able to stick them in one of those big glass cases and just leave them alone.

There are many other instances of Holden withdrawing from society, but I see these as the three most central to his character.