What drove the speaker around the world in Bernard Shaw's poem "Imagination"?

This is an interesting poem. Viewed superficially, it appears extremely childish and simplistic, the sort of poem a child might write for a school assignment. The speaker's comment in the final stanza—that having become a serious adult, the suggestion that he should leave his childhood behind "nearly drove me round the bend"—gives us some inkling as to why Shaw may have chosen this structure and style. The speaker, we may infer, wants to return to his "childish world," and has put himself in the mindset of a child, using childish language, in order to try to return to it.
As a child, the speaker says, it was his imagination which took him around the world. In his own mind, or "fantasy," he became "a Hero of all the Nations," his travels sparked by "the joys of reading." While his imagination was always strong from a very early age—the first stanza describes how the speaker used to "play Pirates" and pretend to be a cowboy—it was in books that he found access to more specific places and times for his mind to explore. Jules Verne, Mark Twain, and stories of Dr. Livingstone took the poet from Africa to America to the moon, with nothing to suggest he should not spend his time daydreaming in this way.
As an adult, however, this world of the imagination was to be put aside—the speaker had to become "serious." The structure and tone of the poem itself, however, suggests that the speaker is now rejecting this confining notion and wishes to return to his world of imagination once more.