I just went through buying a new computer. I'm hardly an expert, but I boggled at scores of sites' recommendations and such for months before I finally was sure of what I was going to do. I'll save you a little time and tell you some things I figured out.
Spoilered for size.
It's worth looking into components that you won't have to replace later on. If you have the budget for it, it's worth getting a nice, big case and decent CPU cooler (If you're planning on OC'ing, that is - If not, stock coolers will do you just fine), since those will probably be as useful five years from now as they are today, and they're one of the few things whose price and performance won't change dramatically due to advances in computing.
SSD's (Solid State Drives) are nice and their quality has risen above the initial problems they had (Namely size and degradation), but probably not worth it if you can't afford one. You can easily get one of the smallest SSDs on the market for the price of a 1TB HDD. SSD's are mostly useful for fast loading times, fast file browsing, and quick boot ups (Also, you don't have to defrag your hard drive), but are largely useless once everything's loaded in the RAM. The price of SSDs will probably be going down in the next few years as the tablet market rises, though, so it may pay to hold off for a while.
Speaking of RAM, you probably won't need more than 8 GB, but at least get 6 GB if you want to run a wide variety of games without too many issues. More than 16 GB is overkill unless you're into video encoding and Photoshop files with a ton of layers, and many will argue against even getting 16 GB. Whatever you do, spread the wealth as evenly as possible - Most motherboards have four slots for RAM, so filling all four up with 2 GB sticks will generally be cheaper and more efficient than two 4 GB sticks. The only upside of using a smaller number of larger sticks is giving yourself room to upgrade. I have no clue how different sizes and brands of RAM interact with each other, though, so I can't recommend that too much, since you never know if something will be going off of the market soon.
As mentioned, CPUs aren't as important as GPUs if you're gaming. Don't get anything you can overclock unless you plan on investing in a decent third party cooler and actually overclocking the guy, else you're just paying extra for nothing. Intel has some good chips out right now, but I can't recommend anything AMD unless you're a loyal fan. Their offerings haven't been too exciting lately, and Intel chips are generally a better deal.
You don't need anything crazy for a Motherboard. Just be sure it has a 3.0 PCI slot for your GPU, fits your CPU, and can hold the RAM you want to stick in it. There's plenty of good offerings in the $100 range. If you want extra doodads, such as space for SLI, you'll pay for it. If you plan on overclocking, be sure that your motherboard supports OC'ing your CPU and RAM separately and is easy to work with. It takes a lot of the hassle out of overclocking, especially if it's your first time.
Power Supply Units are pretty easy, and best chosen once you know what you're getting for everything else. Most midrange systems won't need more than 700W - 800W, if that, but use a few PSU calculators with the build you're going for so you have a better idea of what to get. You don't want to overpay, but you don't want to find that your PSU won't power even a small upgrade. Another important thing to note is that these are another one of those long-term investment pieces: Unless you get unlucky, they can survive at least a couple upgrades to your core system.
If you're planning on doing surround sound (7.1 Dolby and the like), you'll need a sound card. Otherwise, don't bother. The sound drivers for the motherboard will keep you plenty happy.
Cases are pretty easy. Long and tall enough to hold your motherboard, wide for your GPU, and with enough slots for whatever drives you'll want to stick in the front. You probably want a design that has the PSU on the bottom and back of the tower. Don't get any of that half-sized crap, you'll end up regretting it fast. Look for good opinions on the air flow and cooling and you'll be sitting pretty. Remember to measure out the size of one you're looking at so you have a good idea of its size before you order it. These things can get monstrously big.
The GPU is probably the most complicated choice you're going to have to make, and also where the biggest performance boosts will be as far as 3D gaming goes. Generally, you want more CUDA cores. Both AMD and Nvidia have good reasons for you to choose their brands, but AMD is often plagued by poor frame rates in some games, better frame rates in others, while Nvidia is fairly balanced and comes with fun stuff like PhysX processing (Though, unless you've got a fairly beefy card, you probably won't want to use it on most modern games). AMD is generally cheaper, too, so there's also that.
I don't recommend SLI. If you really want better all-round performance, you'd be better of just buying a single card for the price of the two SLI cards. This is the same across the board - Simply buying a single chip is better than buying two. You'll get more FPS in more games, and won't have to deal with wonky SLI issues or the problem of getting a larger PSU to support them. Sure, it looks nice and sounds powerful, but considering the number of games that support SLI are rather limited (Mostly non-AAA games), just get one good card.
An important thing to note: Nvidia is coming out with their 700 series GPUs later on this year (Rumors pointed to this month, but I wouldn't be surprised if they came out in November). While these expensive pieces of plastic and copper may not appeal to you directly, with the cheapest floating at $300 (About £200), the resulting price drops on all of the old stuff will. This is mostly a waiting game, though: Are you willing to wait for the release of the 700 series and the resulting price drops, neither of which have sure dates? It's up to you, ultimately, but it's worth keeping in mind when buying a new GPU.
The single most expensive piece of hardware in your gaming rig will probably be the GPU, so do your research. I don't suggest you go above $300, even for a high-end rig.
I didn't really get too much into monitors and such when I was doing my research, since I'm not getting one, but remember that 60Hz = 60 FPS, and 120Hz = 120 FPS. If your system can't pump out more than 60 FPS on the games you play, then it won't be worth getting the 120Hz moniter. Larger is generally nicer, but I recommend a decently sized 1080p monitor with a low delay time (Generally shown in milliseconds; 2ms is great, 10ms is poor).
Good luck, and don't feel pressured to make a purchase! It took me nearly five months to finally come to a decision I was happy with, and I ended up getting some good deals because of it. If you're really patient, you might be able to catch some of the pre-Black Friday deals in the winter and save yourself a few pounds.