Why is Noda's identity as a Japanese-American woman the most difficult part of herself to accept?

Mainly because from any early age, Noda's whole identity as a Japanese-American is the subject of a mixture of incomprehension and stereotyping from white society. She wears an identity that people simply don't understand, one steeped in a history that has traditionally presented Japanese-Americans in a negative light. To other people she'll always be a stranger, other and apart. In support of this contention, Noda maintains that a third-generation German-American is regarded as an American, whereas a third-generation Japanese-American is just Japanese.
The central paradox at the heart of Noda's identity causes her to consider which is the best way to know and define oneself. Growing up, Noda felt very keenly that her identity was defined from the outside, as it were––from society, the media, and people woefully ignorant of Japanese history and culture. However, there is an another way to conceive of one's own identity:

From the inside—within a context that is self defined, from a grounding in a community and a connection with culture and history that are comfortably accepted...

And as an adult, Noda has been able to see herself and her heritage from both perspectives, moving gradually away from how things appeared in childhood, when her identity had been imposed almost wholly from without.