Rocky II is not so much a rematch but a remake, a bit like the way Evil Dead 2 was a remake of Evil Dead.
The action picks up from the end of the first one (as will further sequels) and then the story just repeats itself. Instead of Rocky being a washed up bum, he's a retired one off super star who vows never to box again. He still can't earn money. Adrian is still whiny. Paulie is still a dick. And Apollo Creed is still overly frustrated with everything. Cue the Rematch.
Of course, none of this is bad as Rocky vs Apollo is the fight to watch, and the music still kicks ass and inspires you with every watch.