Creatures that are made of non-solid matter (eg water, steam, or ash), such as some generated forgotten beasts and titans (and clowns), tend to be extremely fragile because the materials are, obviously, not very tough. This makes them a bit underwhelming for their status, so I suggest they be made more intimidating, by taking inspiration from how non-solid monsters work in other media.
I think these sorts of capabilities below would make creatures made of non-solid materials much more dangerous, and more importantly, fun/interesting. Note that some of the below could also apply to non-solid creatures that come in the future that aren't as strong (like some minions created by a wizard or something).
DefenceAmorphousnessWhat makes monsters like this in media non-fragile? One thing is that they are amorphous! This doesn't just apply to shapeshifting/shapeless blobs; even monsters with defined shapes, like a dragon made of water for example, tend to be a little "flexible" in fantasy. Sure, a sword or spear will pass right through a water or fog monster with ease, but their body is just temporarily displaced around the weapon/doesn't retain damage; compare stabbing a piece of dirt/tree with trying to stab the water in a lake, or chop off a tree branch vs chopping smoke in half; no lasting harm, and whatever temporary disruption/splash there is all comes from the blunt force, increasing with contact area (ie the edge does nothing to them).
They notably also don't have any organs, so you can't really hurt them by removing their heads or targeting specific points for example, and instead of walking around with a missing body part/limb when they lose one, they often just reform it in short order, shrinking the rest of their body in doing so (in principle you could maybe shrink them down small enough to trap in a jar and keep as an angry pet).
You can't really "grab" them either exactly.
Regeneration/AbsorptionAnother thing non-solid creatures can do in fantasy is that, even if parts of them are separated/whittled away by sufficient physical force, they can reabsorb these parts if they touch them again, or might even be able to rebuild themselves with/absorb
normal matter of their type, eg, a water monster being "healed" by the rain, or by dipping into a lake, or a sand monster sucking up extra sand out of the ground, like this Spiderman villain:
https://www.youtube.com/watch?v=43si9s7ulU8WeaknessesI wouldn't be too worried about these abilities making them invincible; firstly, the magic update would bring various kinds of magic to probably offer more options for dealing with them, like turning parts of their body into another state of matter by freezing them (eg, freezing water solid, or chilling steam into being water) or burning/boiling them (turning water to steam, or melting snow into water), or using powerful winds to scatter smoke-like beasts everywhere (or ones made of sufficiently lightweight powders, like ash or something).
And even if you have no magic, regular fire, or magma, can be used instead; freezing is less feasible, but maybe steam based beasts could be whittled away slowly using nether-cap to simultaneously turn bits of them into water as you swing away? This probably wouldn't work for anything else due to the ambient air temperature heating them back up faster than you can freeze them (and may not even work for steam).
Wind could be utilized with likes of fans or billows for the smoky or steamy beasts (maybe as traps/mechanisms, or even as handheld tools that dwarves can use as weapons in this scenario), which could be used to blow parts of them away, or better still, suck parts of them into containers (so they can't reabsorb it).
Water could also be used to dissolve dust-based creatures, see the above youtube video for an idea of how water could impact for instance, a sand-based beast; it makes them clumpy/heavy and easy to break apart, if not dissolving it outright. Water could also
maybe dilute liquid beasts (if they aren't made of water already), though the possibility of them instead corrupting the water (if there's not enough of it) with their bodies should be considered as well. Apparently smoke turns into lye if splashed with water, so maybe that could work too, and fire being extinguished goes without saying.
Fire-based beasts could also maybe be smothered with ash (fire needs oxygen). With water based beasts, or arguably other liquids (which are often partially water), you could also make use of substances like gypsum powder or alum to screw up their bodily composition (I don't know about all the other substances creatures can be made of, but I'm sure similar reactions could be used to mess them up as well). This could work in reverse as well in case there are any gypsum powder beasts roaming around. If you can keep them away from the beasts after being used (to stop it being reabsorbed), sponges could also work for the liquid ones.
Finally, apart from all the logical, scientifically based weaknesses of their bodily material, don't forget that their nature as a clown or mythical beast might entail procedurally generated, symbolic weaknesses, kind of like how vampires in fiction are weak to crosses, or werewolves to silver, even if that logically has nothing to do with their physical makeup. This shouldn't be necessary for all of them though, just the ones where it makes sense to have a symbolic weakness, like with clowns, or spirits representing some sphere, or perhaps if some god is opposed to the creature, something representative of that god could harm them, like their "symbol", or a material they are associated with.
OffenceGoing back to making them more dangerous, offence could be handled well enough with the classic ability to envelop/suffocate with their bodies, and maybe polluting water if they're made of liquid; the myth/magic update will probably take care of the rest, since I expect most creatures like this (ie clowns, forgotten beasts, titans) will have some form of magic due to their origins/nature. As far as magic goes, anything that imbues their bodies and any residue/moisture/dust that comes off them with special effects will be highly effective (imagine for example, a slime monster that leaves a trail that sends people to sleep and digests them).
MovementAmorphousness has a fun property, it means you can squeeze through tiny gaps, so these kinds of creatures could do fun stuff like move through cages, grates, and fortifications (any tile that lets liquids and miasma through is game); even when multi-tile creatures are added, large amorphous beasts could potentially squeeze through tight spaces.
Cages won't contain them at all, but since glass terrariums and aquariums presumably have no bars, those could still contain them. If they're small enough maybe you could even trap them in smaller containers (like glass jars), or just for fun, move them from a terrarium to a barrel! Large pots/other containers without lids likely wouldn't be useful for containing them though. Dust beasts presumably could be trapped inside bags, just like sand.
Ideas from others belowDF already has a model for the contact area of weapons. It would be interesting if amorphous monsters were completely immune to edge (slash/pierce) damage, and took damage from blunt attacks relative to contact area. I think that would be an interesting challenge, because usually you want to choose blunt attacks based on lower contact area. So with a sword, your most effective attack would be a flat slap with a contact area of 20000.
But, indeed these monsters should have some kind of insane healing ability, so that they reconstitute body parts in just a few seconds. After all, there's some magical force obviously holding this amorphous mass together...and the game designer can decide how strong that magic is. I would also guess that hitting them in the head ought not to necessarily incapacitate them, and that instead some percent of their body would need to be disrupted at the same time for them to die.
It should also be possible to research ways to counteract the beasts. For example, scholars could discuss and develop a glue-dust-launcher that is effective against water bodied monsters. Or a system of fans and vents that can disperse smoke-monster bodies and make them smaller. Creative methods for creative foes.
the concept of them getting smaller but not losing limbs also sounds awesome. Like, eventually they drop below a threshold and either die or become vermin/passive and can be put in a pot/jug/glass terrarium they can't escape from.
I'm pretty sure they're already meant to be nearly invincible, and that it's just a lack of coding that means gas monsters are flimsier than paper. That is, the combat system just isn't built to handle any material that isn't a solid, and to treat shattering a solid as the worst thing that can happen to it. See also the early versions of mine carts where minecarts that hit water would skip and "splash" water that was treated as a solid. Or, in other words, it allowed the creation of "wave cannons" that flung high-speed icicle projectiles at invaders as a weapon. This was nerfed a few years ago with less fatal water splashes.
Granted, there's also a lack of ways to fight a creature made of smoke you can't stab, so adding in dwarven billows brigades that try to suck up all the smoke FBs would need to be part of the update, lest we be forced to fall back on "just don't fight them and use magma, instead".
Water: If this were a JRPG, I'd say lightning, but in this game, something like alum, gypsum powder, or sponges would probably be hilarious suitable.
Steam: Chill it to death - using nethercap wood weapons is a good idea. Alternately, using fans or billows.
Smoke: Billows or using water (which turns it into lye).
Fire: Water or cold, including the above nethercap wood weapons. Alternately, smothering it with dust so it asphyxiates.
I don't consider the specific matchup of currently existing FB materials to weaknesses to be nearly as interesting as creating a system that allows for procedural monsters to have procedural weaknesses and the system by which players can counter it being implemented.
Something like the bogeymen being destroyed by daylight or even being kept at bay by fires you light is a good example of a creature that has a mythic bane, and so are classic weaknesses like silver against werewolves or the many weaknesses of vampires.