I've also got a frozen river in the current fortress. My solutions:
a) dig a basin/reservoir underground and fill it before winter arrives or
How exactly would I go about building a reservoir underground?
Firstly: it's too late to get any water into an underground reservoir when the water has already frozen, so it's a task that prepares for the next winter (barring magma heating, but arranging that probably won't be finished before spring anyway).
To build an underground reservoir, you just dig out an area that's at least two Z levels deep (can have sloping edges) to avoid muddy water. To fill it you can either use a bucket brigade (designating a lot of pond zones over the edges of the reservoir to get dorfs to fill it one bucket of water at a time). Obviously, that requires a source of liquid water for the dorfs to draw from (and designating a water zone can be used to direct where they take it from).
A better alternative in general is to dig a tunnel from the reservoir to the bottom of the river (and if that's done during winter you can use the ice to your advantage, and you don't risk the miner drowning). The tunnel should have an airlock (two raising drawbridges with a space in between) so you can ensure no nasty critters get into the fortress through the water. If you do the digging when the water is liquid, you can build the tunnel up to the edge of the river, build the airlock, and then channel out the last tile on the river bank to let the water through.
As Loci mentioned, disabling the hospital and deconstructing any racks dorfs are hooked up to should release them to get them to get drinks themselves. Once they've drunk their fill, you can restore the hospital and rebuild the racks for a new session. If the dorfs have lost the ability to use their hands you need to forbid all mugs or they'll sit holding (somehow) a mug in front of a barrel and just stare at that lovely booze the can't get at. With no mugs they'll just uncork the barrels with their teeth (or something) and sate their thirst.