Why can't Jem accept Scout's opinion about people?

At the end of chapter 23, Jem and Scout discuss Maycomb's caste system. Jem tells his sister that he believes that there are basically four types of folks in Maycomb. He goes on to tell Scout that there are regular, educated townspeople like their family, poor but respected farmers like the Cunninghams, despicable white folks like the Ewells, and then the black people at the bottom of the caste system.
Scout challenges her brother by pointing out that Atticus has similar traits as the Cunninghams, which doesn't make them much different from their family. When Jem mentions that what makes people different is how long their family's been reading and writing, Scout responds by pointing out that people like Walter Cunningham, Jr. simply lack the opportunity to learn. Despite Scout's arguments, Jem cannot accept her opinion that "there’s just one kind of folks. Folks" because he is jaded after witnessing racial injustice firsthand (Lee, 231). Jem explains why he cannot accept Scout's view of people by saying,

"That’s what I thought, too . . . when I was your age. If there’s just one kind of folks, why can’t they get along with each other? If they’re all alike, why do they go out of their way to despise each other? Scout, I think I’m beginning to understand something. I think I’m beginning to understand why Boo Radley’s stayed shut up in the house all this time . . . it’s because he wants to stay inside" (Lee, 231).