"Turtling" comes from the idea of a turtle pulling his legs and head inside his shell for protection. In SC (StarCraft and it's expansion, Brood War) this is using terrain, defenses, and specific units in such a way to prevent your enemy's ease of access into the vital portions of your base such as your mineral line or tech buildings. It is essentially an elaborate word for a common type of defense in SC. Turtling can be very effective at forcing your enemy to use many of their resources to attack you without doing a whole lot of damage to your economy. However, turtling makes it very difficult to go on the offensive and if you fail to turtle properly, you can make yourself very vulnerable. In this regard it can be very expensive if not done effeciently, but with practice, turtling can be mastered with a proper defense/attack ratio that will make your gameplay much stronger.
With turtling, terrain is extremely important. Turtling on the high ground is much more effective than on the low ground. Usually there is a ramp leading to the high ground that must be overtaken and if you can force your opponent's forces into getting stuck on that ramp you can truly own. You also get better vision on the high ground. Don't try to turtle without vision unless absolutely necessary. Turtling in a valley is extremely wasteful. You may feel protected, but if you let your opponent get a high ground advantage on you, you will be a sitting duck, no matter how many cannons, spore colonies, or turrets you have surrounding your base.
Players that turtle will often do so early in the game so as to protect their mineral line. Be careful not to turtle too long or your opponent will seige you until you have no minerals left. Don't forget to expand. This means that some of your defenses will fall and you won't be able to replace all of them. Don't get stuck here. If you can't mine, you can't win the game. The point of SC is not to build the biggest and baddest base, but to win. Blizzard did a good job of designing the units so that there is some unit composition that your enemy can use that can defeat your defenses. If your opponent is smart he will catch on to your strategy and work around it.
Items to add to this page: Specific strategies for turtling as different races.