When you're done in Route #22, make your way back to Viridian City and exit through the north path to reach Route #2. Much of this route is inaccessible to you this early on, but you'll be able to reach it later after acquiring HM 01. For now, your only concern is reaching the entrance to Viridian Forest. Pick up a Pidgey or Rattata if you haven't caught one yet, but don't bother hunting for Caterpies or Weedles, they're more plentiful in the forest.
Caterpies aren't worth getting too excited about. Deciding which of the two bugs to collect comes down to whether you want an awesome Pokémon sooner, or later. If you'll willing to invest a lot of time with Caterpie, he will ultimately end up as a Bug and Flying-type Butterfree that will eventually learn powerful psychic attacks.
Weedle doesn't start out much differently than Caterpies. He's pretty weak when you get him, and evolving him into Kakuna doesn't make him much better. But his evolution continues at a lightning pace, and at level 10 you'll have an awesome (at least for the early part of the game) Beedrill.
Second visit
After you beat LT. Surge in Vermilion's Gym, you will be able to return to Route 2 through Diglett's Cave. You will get to the area which you couldn't get to in the beginning of the game. There's a trade as well as some items that you couldn't reach before.