I’m doing a type of annotation essay/assignment for my English class, but I’m having a hard time understanding what would be relevant when doing annotations within an essay. What should I be looking for when annotating "Crusoe in England" by Elizabeth Bishop?

An annotation is just an explanation or comment added to a text. For example, Bibles are often annotated with footnotes throughout, offering explanations for historical context, translation difficulties/ambiguities, or relationships to other texts. Annotation is also a practice that any reader can undertake. You can take any text and create your own annotations by highlighting, scribbling notes in, or providing notes that point to related information in the same text or in others. 
It is hard to give specific direction without knowing exactly what your assignment is, but presumably you are being asked to annotate the text of "Crusoe in England" by Elizabeth Bishop. Alternatively, it may be that you are being asked to write an essay that leans upon the text and refers to it consistently throughout. Either way, your approach can be the same. You will need to break the text down and focus on specific passages and phrases to support whatever your argument is. 
Are you being asked to focus on the symbolism or themes in the poem? Bishop uses the character of Robinson Crusoe to make points about her own thoughts and concerns; the entire poem is an extended metaphor used to make these points. Crusoe's return to England and the associated emotions represent Bishop's own concern for her inability to produce poetry and express herself through her art. 
Are you being asked to focus on the structure of the poem? It could be described as a dramatic monologue, which includes the following characteristics:
A single person, who is patently not the poet, utters the speech that makes up the whole of the poem, in a specific situation at a critical moment.
This person addresses and interacts with one or more other people; however, we know of the auditors' presence, and what they say and do, only from clues in the discourse of the single speaker.
The main principle controlling the poet's choice and formulation of what the lyric speaker says is the speaker's temperament and character, which must be revealed to the reader in a way that enhances interest in the poem. 
You could also describe the poem as an elegy in tone, as elegies are mournful laments. 
Regardless of the topic, list out concrete points that you are trying to make and then associate them to specific moments in the poem. This will allow you to strongly support the points you are making, and a list of concrete points will keep you focused in your analysis. Some of the links below may help. Good luck!
http://www.modernamericanpoetry.org/criticism/susan-mccabe-crusoe-england

https://seapoetry.wordpress.com/2012/02/08/elegy-exile-elizabeth-bishops-poem-crusoe-in-england/

https://www.modernamericanpoetry.org/dashboard