It does make sense for things dropped into magma to not burn. Oxygen is required for combustion, and while an item is immersed in anything denser than air it will not be available for combustion. Whether Toady did that intentionally or by happy mistake it is still good.
The above may be strictly true, but not entirely helpful. There are several different sorts of heat-driven reactions; not all of them are technically "burning", but when someone speaking casually talks about things "burning in magma", they're probably intending to refer to a broader class of thermal damage than just rapid oxidation. However, it misses some of the real problem.
Magma in DF has a set temperature of 12,000 degrees Urist, which is 2,032 degrees F or a hair over 1,111 degrees C. This is somewhat problematic, as on Earth, (comparatively) near-surface magma commonly ranges from about 700 to 1,300 C, with outliers down to 600 C and up to 1,600 C; and of course it can gradually cool down to rock. (The highest end of the range is no longer found on Earth; very hot and fluid ultramafics such as komatiite were much more common early in Earth's history however.)
DF has an approximation of a heat transfer model to speed calculation, which seems to simplify exposed temperatures based on a rough concept of distance. On Earth, a comparatively small object that remains in close proximity to a comparatively large hot mass will increase in temperature gradually, but eventually come fairly close to the temperature of the large mass; in DF, this would introduce "temperature flow" and kill framerate, so is not modeled.
I've found several different references to the thermal decomposition and/or burning of bone, and they generally agree that much higher temperatures are required to reliably do so. It's generally agreed that crematoria do NOT burn bone; despite common misconception, the "ashes" you get are the bones, put through a mechanical grinder. Various countries have different traditions, but a legal minimum of 850 C (to decompose various gasses) shows up in various places; Wikipedia describes the common range as 760 C to 1,150 C, with typical times of one to over three hours. At least one forensic site describes temperatures of 1,100 to 1,500 C as insufficient to reliably destroy bone. The main refractory component of bone is calcium phospate, specifically hydroxylapatite, which is about 50% of the bone mass in life; ground hydroxylapatite is described as having a melting point of 1,670 C. Calcite, a generally related calcium carbonate, has a DF melting temperature of 1,613 C, which is generally compatible with various other references (different forms have higher or lower melting points).
In common situations, much of the damage to bone from fire is from the steam explosion of the "goopy bits", which tend to at least crack it if not shatter it into fragments. A recently-dead body thrown into a hot fire or exposed to DF magma temperatures should not remain intact, but should not evaporate either. A "cured" or aged skeleton, either due to the deliberate act of morticians or necromancers, or the passage of time and elements, will have far fewer problems of this sort, and if particularly dry (long storage in a dry mausoleum or pyramid) may not take any significant damage to the remains from heat of this level.
tl;dr: Live bone is made up of about half magma-safe materials, but due to steam damage a fresh body exposed to magma should leave only chunks of bone. Well cured skeletons on the other hand should actually be more reliably DF magma-safe than many materials routinely used in DF to contain magma, such as iron (melting point only 1,538 C).
The results one expects from exposing undead to magma temperatures therefore strongly depends on the magic used to make them undead, which is not yet modeled in any detail as of the current release. I will point out that the telekinesis-like effect is strong enough to not only hold together the bones of a skeletal elephant (or whale) in the absence of all of the connecting tissue that would normally do the job, but to do so under combat conditions; it is also capable of propelling skeletal eagles through the sky in the complete absence of any physical (non-magical) lift system.
My personal take is that there should probably eventually be several types or "strengths" of necromantic animating force.
At the first level, zombies are still mostly functioning as they were in life, with some other process replacing a few traditional life processes (mainly circulation and energy production). This would cover the simplest of necromantic magic, as well as "zombies" that are actually created by fantastic bacterial infections, mold spores, or whatever. They would generally no longer take bleeding damage, but ordinary physical and thermal damage would work pretty well. Burning one would leave inert bone chunks little different from that of a non-animated body.
At the second level, zombies are held together with some force beyond their rotting remains, but the bulk structure of the corpse is still relevant. There might be rare physical explanations for this (the most exotic of colony organisms), but generally this would be the next major advancement of necromantic magic. Gross physical damage will reduce capabilities, but may not be sufficient to "kill" it, which may require a special focus location to be destroyed (the brain, the heart, etc.); at the high end, merely severed parts may simply become new creatures (hand-spiders, leg-hoppers, etc.). Surface burning may not be very effective anymore, but chopping combined with burning or magma immersion should leave inert bone chunks. This level may also encompass mummies, where supplemental structural material has been added; the question then becomes what are the weakness of the added reinforcement (fire is obvious for traditional linen-wrapped mummies... but how do you get rid of a legendary mummy wrapped in adamantine cloth?
At the third level, the necromantic force is much stronger and no longer requires the shell of flesh; the magic is stronger than flesh, sinews, cartilage, and so on by now. While most commonly encountered as skeletons, it may be possible to produce this sort of creature while flesh remains, leading to what appears to be a zombie but is really closer to a skeleton with ablative flesh armor. Depending on the sort of magic involved, they may have a mystically-significant "critical location", or may require total dismemberment to put down. If a recent corpse or properly treated, magma immersion may cause enough physical damage to produce sufficient dismemberment; a well cured skeleton may well be impeded by magma (it's heavy and thick), but possibly not stopped.
At the fourth level, the necromantic force is more powerful yet, and capable of reanimating from bone chips or possibly even dust. This may result from ever more powerful animation spells, or because the undead is actually something closer to a ghost with the ability to animate its own remains. Magma will not help you here, nor will traps nor axes, unless the magic has an explicit weak point flaw of some sort you can attack. You would need the ability to attack the real structure of the creature / construct, which by this level is fundamentally a structure of magic and/or spirit. Presumably later magic arcs will develop this; it is possible (hopefully likely) that special materials (e.g. orichalcum, commonly thought of as a gold-copper alloy, has such properties in some mythologies) or treatment (e.g. rune-crafting as an added decoration / encrustation type) will allow interacting with the normally-intangible.
Heat and cold damages not affecting/killing undead ceatures is still a bug that should be corrected.
This is strongly a matter of choice of mythology and to a lesser degree opinion; in general I disagree. The primary effects of heat and cold to living organisms made mostly of meat are irrelevant to all but the weakest of zombies, as those processes have already been replaced with whatever the necromantic animating force is. Gross physical effects such as freezing in a block of ice or encasing in magma should be dangerous to all but the more powerful undead, but just because a dwarf is 90% water and tends to freeze on a glacier or melt near magma doesn't mean a skeleton should be slowed down at all. (I will point out in passing that in reality there are several mammoths and even a few humans that have been discovered after being frozen in glaciers for thousands of years, in good enough condition that they could easily be zombies.)