Every item in Oblivion has a certain weight, intended to limit the number of items you can carry at once. However you can't leave them in containers, because their contents get reset over time, and you can't drop them on the ground or in a building because NPCs might take them. So what's an adventurer to do? The answer is buy a home, your own little corner of the world to store items in and hang out. They're expensive, but it's totally worth it to have at least one.
Items in your house will never be lost by container resets or picked up by an NPC, so you can decorate your home however you like it and it will stay that way.
Imperial City Shack
Ironically, the cheapest and smallest home is the one in the biggest city in the game. A mere 2000 gold will net you this house. This is actually just a shack in the waterfront. It's an ideal location for early Thieves' Guild activity and very close to the big city.
How to Obtain
Talk to Vinicia Melissaeia in the Office of Imperial Commerce in the Market District. Ask about "Buy a House in Town" and she'll tell you about the lovely little shack. Accept the charge of 2000 gold and the home is yours.
If you want to furnish it then you'll have to buy furniture from the Three Brothers Shop (in the market district.) It's 732 for each piece if you don't haggle (3660 if you totally pimp out your shack).
The Anvil House is the only house where you have to quest for it rather than pay a large fee. As a result it will take more time and fighting, but will be cheaper.
How to Obtain
Start by speaking to Velwyn Benirus, you can find him at the Count's Arms if you go there between noon and midnight. He offers to sell you his grandfathers mansion for 5000 gold. Agree and he'll give you the deed and key.
The moment you enter or sleep you'll be attacked by 3 ghosts. Take them out with an enchanted weapon or magic and move to the basement. You'll see what looks like an urn, a skeletal hand, and a diary. Take the hand and the diary.
The diary says that the grandpa Benirus was a necromancer obsessed with becoming immortal and only his heir can open the creepy looking dream catcher in the basement (apparently it's some kind of door.) Head back to the Count's Arms to talk to Velwyn.
Turns out Velwyn bolted on you. Other patrons say he left town as soon as he sold you the house. Looks like he knew it was haunted and sold it to you anyway. Track him down at the Kings and Queens Tavern in Imperial City. When confronted he agrees to help you and will meet you at Count's Arms.
After meeting him there he'll ask you to follow him to the house, once inside you'll be attacked by a total of 7 ghosts, two on the ground floor and five in the cellar so be prepared. You do not actually have to fight all the ghosts in the cellar, just move straight to the seal and defend the area while Velwyn opens the seal, after it opens, the ghosts will go away.
Velwyn opens the door to his grandfathers tomb and then runs back to the bar. Head inside and talk with the grandfather.
He'll tell you that if you connect his hand he'll be able to rest in peace. Connect the hand and he'll rise as a ghostly specter. Kill him and loot the room. Leave the house and come back later for a cosy house.
This is by far the most expensive house in the game, it is in Skingrad.
How to Obtain
You will have to talk to an orc in the country hall, you will need a fairly high disposition then he'll tell you that you can have the house for 25,000 gold, if you pay the sum, you can go to the store and buy furniture, there are 13 pieces for 2,749 gold, 1 for 2,405 and 1 for 1718, this adds up to 39,860 gold for all of that. Including the house, you will pay 64,860 to fully furnish the house. Though should you show up at a economically correct time (random) the prices will be cheaper. You may also lower the price by becoming friends with the seller.
Note: You can find a quest on the top floor on the roof if you jump on the desk then the moose horns then the roof part.
You will find a note that contains a riddle.
To solve this riddle (and receive your treasure) simply head down to the basement of the house. There on top of one of the pillars sits a hourglass, inside lies your treasure!