For that price, you could probably build a desktop capable of running most games on high settings. How much is spent on each part would depend on various things like how much you want to future proof it. Assuming you are going to use Windows 7 as your OS, may as well set aside about $100 from your pool for that, leaving you with around 1000 for the hardware [even using a free OS like Linux, you're probably better off trying to spend $1000, as you will probably find something which you really want for just a little bit over the budget you set for yourself].
A video card costing about $200 should be able to run just about anything on high settings with acceptable framerates; be sure to compare cards and their reviews/benchmarks before buying, though for that price you typically want the high end card from a generation or two behind the most recent generation of cards.
Power supply and motherboards should be chosen to match the connectors for various other parts as well as how much future-proofing you want in your system. For a power supply, get one rated for much higher power consumption than your system, as the lower a percentage of it's maximum you use, the longer it will probably last; the higher wattage, slightly more expensive varieties also tend to catastrophically fail (taking half your components with it) much less often than the cheap ones. If one comes with a case you buy, you should almost always toss it for one bought separately, as the ones which come with are often pretty low quality, and thus prone to failing. For the motherboard, be sure it is compatible with all the stuff you need, and depending on how much future-proofing you want, more advanced parts and connectors as well. These two combined should probably be about $250 ($100 or less for the power supply, somewhere between $100 and $200 for the mobo).
A good CPU can be found for about $150. By this point, you should decide whether to go with AMD or Intel; when I bought mine, AMD were a better price/performance for this price range, but be sure to research which to go with. Not sure if they're just hearsay, but I have heard rumors about AMD and Intel parts not working as well together than AMD/AMD or Intel/Intel, so you may want to consider buying both mobo and CPU from the same company. Personally I prefer AMD, but personal preference is mostly all it is.
The main purpose of the case will be to keep the components cool with secondary duties of keeping things organized, dust free, and looking bad ass to visitors. A mid tower should suffice, and these can be acquired for around $50-$80. The main thing to look for here is good airflow and durability of said airflow, noise can be a factor, but unless something is broken, the fans typically won't make too much noise.
4GB DDR3 memory should be sufficient; any less to too little, any more than 8 is overkill (even 8 is more than you need for just about anything). Be sure the timings match up [with the CPU, IIRC]. This should be under $100.
For the hard drive, I'll leave it up to you as to your priorities for that; IIRC, faster drive means shorter load times, and of course larger drive means more storage space. Though you will want the typical hard drive rather than the (at this point) prohibitively expensive solid state drives. Well under $100 here; and most modern components do support multiple drives if you need to add more space in the future. And with the rapid price decrease in hard drive space, you may even be better off going for relatively low storage space (for example around 250gb) and simply buy more when you need it at a much reduced future-price.
CD drive is up to you; I went with the cheapest I could find (under $20); you should probably do about the same unless you plan on writing to CDs (mostly pointless in an era of flash drives and ubiquitous internet). Just so long as you have one, you don't really need a good one.
Keyboards can be acquired on the cheap. I got mine from the local supermarket for $7.50, and not only has it survived for several years, but it is a pretty nice one as far as keyboards go.
You can get a good gaming mouse for about $40, possibly less, though this is mostly dependent on what type of mouse you prefer.
You can either go speakers or headset depending on where you will be using it (I use a headset, as I live in dorms and with other people around). Again, quality is dependent on preference; I personally get $15 logitech headphone/microphone set at Walmart since I'm not anything resembling an audiophile.
What monitor you will want will depend mostly on where you plan on putting your computer; an LCD monitor probably shouldn't cost you any more than about $200 unless you really want to go high end.
I think that's just about everything; you may have noticed the list adds up to more than the $1000 for hardware, but it's more of a starting point to work off of. I highly suggest making a list of parts and checking through thoroughly for compatibility before buying anything; and from there, replacement of parts based on prices can be done. newegg.com is a great place to check out if you are willing to build your own system as described above; especially because of their really nice return policies and customer ratings.