A few things to note (spoilers around longest descriptions, but they aren't too long):
Caffeine is not very addictive at all, unlike cocaine or heroin. Beer and marijuana aren't addictive either. Cigarettes, however, contain nicotine, and that is where most of the controversy lies.
Cocaine is a powerfully addictive stimulant. Snorting it carries the usual risk of snorting anything, which is deterioration of nasal membranes. In binges, which are likely due to said addictiveness, it has been known to cause irritability, restlessness, and paranoia (the latter of which can develop into paranoid psychosis, which causes the user to lose touch with reality entirely.)
LSD can also cause permanent psychoses, and it stays in your body for ages, so it is in fact a dangerous drug.
Crack is basically crude cocaine diluted by cooking it with other stuff; usually baking soda or ammonia. It's basically the poor man's cocaine. Even taken only once, it causes a significant increase in risk for heart attack, stroke, respiratory problems, and severe mental disorders. It is basically the result of people taking cocaine and trying to stretch it farther to make more money, which could easily happen to the cheaper illicit drugs if they were made legal with a heavy tax.
Ecstasy is a stimulant, which are generally speaking only handed out with prescriptions, due to the risk of heart failure associated with stimulants. It will also slowly destroy your spinal cord and cells that regulate hormones. In the case of spinal damage, it doesn't matter how well you moderate usage; damage to those nerves does not heal. In the short term, it messes with your senses and causes nausea.
Methamphetamine is highly addictive and causes violent behavior, anxiety, confusion, insomnia, paranoia, auditory hallucinations, mood disturbances, and delusions in the long term. When it is snorted, it deteriorates the membrane of the nostrils, causing nose bleeds. Smoking it causes severe respiratory problems. Fun fact: Hitler shot up on meth 3-5 times a day.
Heroin is highly addictive as well. It is considered by some as the most abused and most rapidly acting of the opiates. When taken by injection, one develops collapsed veins in the area most frequently injected. Standard long-term effects of usage include pulmonary complications and, when the heroin is impure, death by poison (which would still be an issue due to the not uncommon, not common business practice of mixing other stuff in to get more money for less stuff. This happens in most food industries, and occasionally cigarettes). What is easily the worst part of heroin is withdrawal, which occurs within ours of the last dose, which is a major cause of its addictiveness. Heroin is also sold as "cheese," which is heroin mixed with crushed up sleeping pills. "Cheese" isn't much safer. At all. People have died from it.
Marijuana, when smoked, produces the same long-term respiratory complications as cigarettes. However, it is far less addictive than nicotine, greatly reducing this threat. Medicinal marijuana normally isn't smoked, as there are several safer ways doctors know of to administer such things. Either way, prolonged use of marijuana (or the active ingredient, tetrahydrocannibincl, or THC) will cause loss of short term memory capability.
I got all this information from my AP/IB Chemistry, Law Education, and Health course notes (the latter is admittedly a bit out of date with its textbooks, but I wasn't looking through that textbook just now), and it can be confirmed by Wikipedia, among other (likely more reliable) sources, with the exception of "cheese," which I heard about on the news (CNN, I think).
Just for emphasis, quitting smoking is small potatoes compared to most of these other drugs mentioned.