Why did Pip return home the day after Joe's visit to London?

When Joe came to visit Pip in London, Pip was unfriendly and rude to him. It wasn’t until the end of Joe’s visit, when Joe passed on a message that Estella wanted to see him, that Pip starts feeling a little better toward Joe. Joe is amazingly forgiving of Pip’s behavior, saying in his own way that they have grown apart and sometimes that happens in life.
Pip does feel bad about how he treated Joe, and tells himself he must go and apologize the next day. He takes the coach to his little childhood village, but his real motivation is to see Estella. This is shown when he decides to stay at the local inn instead of with Joe. He has plenty of time to see Estella and the various people gathered at Miss Havisham’s house.
In the end, Pip doesn’t even see Joe. He returns to London and sends him a codfish and barrel of oysters to apologize for not stopping to see him when he was in the village. He was so overwhelmed by seeing Estella and the events that happened during his short visit that he ignored Joe.