My dad has a special mat (you put the computer on it) that is supposed to stop static electricity. It comes with a line that attaches to your wrist so you're always at the same potential as the computer.
My boyfriend doesn't use a mat, but he's very careful to always be in contact with the metal case. Usually he just leans his forearms on the case while he's working on it, making sure to touch the case before the computer parts.
Static electricity can be a pretty big problem with computers. My dad was saying he's seen some employees of computer stores working on computers and not taking precautions - I don't know if it's that they don't know or if they just don't care (as if the computer fails, it means the person will bring it back and more business for them).
I was trying to instal the memory on my parents' computer myself last week, but I was too scared I was pulling and pushing too hard, so my dad took over.
Just a basic machine should be easy. If you don't have a case, that should cost maybe $60 (for a pretty one

, I got a 120GB harddrive for $62, 128MB video card for $30, 512MB RAM for $60. Motherboards are a little more expensive - maybe $100? - and same with processors. I don't remember how much my boyfriend paid for his last motherboard and processor, so I can't help you there, but I do know he got the best that the store had.
If you already have a monitor, you don't need another. Same for keyboard and mouse. So you'd just need the actual CPU parts and then an OS. If you're not going to use your old computer, you can call Microsoft and have them switch your license to your new machine (I believe) if you've bought Windows, they won't do it if the OS came with the computer when you bought it. However, you can get Windows for $100. Linux is good, but some programs and games still don't support it. I've never tried it myself, though (but always wanted to!).