The biggest problem is that the human body is remarkably resistant to death (thankfully). Explosives are the standard means of inflicting maximum damage in minimum time, however they have the problem of collateral damage. If we're not worried about collateral damage, hey, tactical nuclear detonation for the win!
By contrast, a paralytic nerve agent delivered via inhalation or dermal contact would inflict maximum human fatalities whilst avoiding structural or environmental contamination. Remember, this topic is for theoretical weapons, not necessarily ones that currently exist (at least that the general public is aware about, dun dun dun!) so it should be fine to hypothesize on our ideal nerve agent.
For our theoretical compound, I'd anticipate something like either
VX gas or
botulinum toxin.
VX gas has the benefit of being a simple chemical compound. The molecular structure is primitive and readily synthesized. The downside is that it's fairly good at remaining in the environment it is released into, meaning it's a poor candidate for our theoretical weapon if we then want to send our own troops into the captured space. This assumes we're not using mechanical drones to perform recovery actions of course.
Botulinum toxin is a large protein molecule, thus being far more difficult to mass produce, store for extended periods, and administer quickly. However, it's interesting to note that the overall paralytic mechanism is very similar to VX gas, yet at the same time exactly opposite. Botulinum toxin blocks acetylcholine release from nerve cells and leads to lack of activation, or flaccid paralysis. VX gas, on the other hand, blocks the body from producing the compound that is manufactured by the body to break down acetylcholine, causing build-up of acetylcholine until the muscle becomes overstimulated and completely unresponsive.
In both cases, they lead to paralysis of acetylcholine dependant muscles in the body, resulting in zero response from the nerves that rely on this neurotransmitter to function. This is typically fatal via paralysis of the diaphragm. Total death should occur roughly three minutes after this occurs, however another benefit of targeting acetylcholine dependant muscles is that you'll likely trigger loss of vision and muscle convulsions well before this point, so the unlucky subject will likely be disabled from taking any meaningful actions well before death occurs. The benefit of targeting this neurotransmitter with your chemical weapon is that you can achieve paralysis relatively quickly compared to other means as well, which is why so many natural venoms in nature act on this neurotransmitter.
I'd suggest our ideal theoretical compound should be a either an inhaled toxin featuring a potent acetylcholinesterase inhibitor or an acetylcholine inhibitor. For the former, there's actually some interesting research in medical applications for naturally occurring paralytic venoms called
onchidal produced by molluscs. Note these chemical structures are relatively simple molecules and thus will likely be simple to produce. They lack the phosphate structure present in current nerve agents, and might prove a novel new way to create highly lethal paralytic toxins. For the latter, it would be too soon to predict what sort of compound could chemically recreate botulinum's acetylcholine inhibitory effect, since it's functioning at the protein receptor macro level rather than the neurotransmitter chemical micro level. Some technobabble handwaving would be required here since modern medicine isn't advanced enough to fully understand and refine this process, but the science and mechanism of action is sound.
As for protecting our own troops during retrieval, respirators would be the way to go, and antidotes to both compounds are pretty easy to manufacture even with today's technology provided you administer them in a timely manner.
Edit: A quick note to address a few other points that the OP raised about theoretical weapons in the first post. First, regarding effectiveness. Nerve agents have already been shown to be incredibly potent and effective weapons when used, though they're by nature indiscriminate. This leads us to the second point, international legality. Organophosphate weapons such as VX gas are classed as weapons of mass destruction and are thus explicitly banned and regulated by many international treaties. This is why I suggested the onchidal toxins as an interesting alternative, since they're not technically covered by any international treaties since they don't contain any phosphate compounds. Finally, for cost of manufacturing, the onchidal toxin based gases would be incredibly cheap if you can figure out a simple chemical reaction to produce your base molecule, since there's no large protein structures or complex chemicals involved. Seriously, you can't get any cheaper than carbon and oxygen.