Abusing a small glitch to double jump early (which normally comes just after Nation of Fools) can get you the Nebula (a homing whip), the most powerful whip that is obtained before Vampire Killer note For some reason various other whips better than it but worse than Vampire Killer can be obtained after you get Vampire Killer, a powerful weapon that will last until endgame unless you abuse a Good Bad Bug to buy the Holy Claymore early.Later on, the Vampire Killer itself takes on this role - it's not one of the game's several Infinity Plus One Swords and you don't have to go out of your way to get it (just beat That One Boss), but many players stick with it for its ease of use and handy elemental attributes.
Its high power and speed will get you through a large part of the game.