Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Dwarf_Fever]

I took some time to run a lot of weapons through some tests, and thought I would share the results here.

Important notes:

• All dwarves were in full iron kit, wood shield, a leather cloak, and got a steel weapon unless otherwise noted. This is to simulate the upper end of realistic siege situations. Goblins will tend to cap out at iron plate, and I consider steel a pretty standard weapon material for dwarves to carry. Blade weapons below steel suffer very poor performance facing common armor.
• All dwarven skills were set to grandmaster, including miner for pickdwarves and, importantly, all physical stats were maxed out for the dwarves. This is because I have noticed considerable difference in performance when certain weapons, especially blunt, are wielded by weaker dwarves. (In one previous test, whip win rates went from 14% to 82% when fighting bronze colossi after maxing dwarven stats.) Thus all these tests reflect a late-game fort and military.
• This being the arena, there was no body part resurrection, so take that into consideration when evaluating weapons that are noted for creating a lot of extra parts. (Looking at you, axes, swords and picks.)

Some preliminary tests I ran first:

• Steel mace vs silver mace, using the above parameters. Steel mace won 59:41
• Steel warhammer vs silver, ditto. It won with the exact same ratio of 59:41.
• Next, this led me to compare silver whips vs iron whips, as steel whips are normally unavailable. 100 duels resulted in 52 wins for iron whips, vs 48 for silver whips.
• Lastly, 100 steel axe vs iron whip duels, to see if whips performed extraordinarily enough for me to want to include them in the roster. Axes won 53:47, and since this was an armored test that whips supposedly excel at, I did not add them to the weapon v weapon roster. I included them in other tests to see if they might have a niche there.

Due to the above results, all subsequent tests use steel for hammers and maces, and, where whips are tested, iron.

The first battery of tests involved putting every single weapon type up against every other weapon type in 100 1v1 duels. The chart lists both how they ranked against all the other weapons (#1 won against all other categories, #2 won all but one, etc) as well as their percent chance of winning a duel averaged over the total 2,000+ matchups.

I consider this to be a good evaluation of combat against siegers, who are most often dwarf-sized, living, and come in up to iron armor, while they have to face steel+ weaponry.

Next, I devised four tests to evaluate two other important categories: undead enemies, and large enemies. The dwarves remained as above, in iron armor, wood shield, grandmaster skills, maxed stats and steel weaponry unless otherwise noted. Each of these only received 100 runs, so these results may not be as accurate as the Sieger column.

The first additional test involved each dwarf facing 6 animated corpse goblins with an axe and recording how many dwarves survived, over a run of 100 fights.
The second additional test involved the same 6 enemies, except this time they also have their stats maxed, and are clad in full iron armor, in addition to the axe.
The third test pits the dwarves, outfitted as per usual, against 3 giant elephants at the same time. Again, 100 runs, and survivors are recorded.
The fourth test pits each dwarf against one bronze colossus with his skills set to competent (3).

Results are below.



(Final note: Wrestlers were included in the weapon v. weapon evaluation, but placed last, not winning against a single other category, and having an average chance of 35% of winning a duel. They were not included in any further tests.)

[Panando]

Very nice tests. As a member of the Steel Shortsword Masterace the results are pleasing to me. Though it also has to be admitted that with maxed stat dwarves, the Steel Battle Axe does exceedingly well.

[Dwarf_Fever]

Very nice tests. As a member of the Steel Shortsword Masterace the results are pleasing to me. Though it also has to be admitted that with maxed stat dwarves, the Steel Battle Axe does exceedingly well.

I happen to like steel shortswords for my citizen militia, myself - as a blade, you don't need huge strength to do damage, and they're good generalist weapons. Soldiers tend to get more specialized weapons, spears, axes or sometimes a few hammers for smashing undead bits.

Speaking of axes, I saw your testing, and found the differences very interesting. In particular that axes did so poorly against a giant elephant, while they excelled in the tests I ran.

I suspect that it's not just the strength, but also the material of the axe when trying to effectively chop into a beast as large and tough as a giant elephant.

The contrast also seems to highlight that, while blunt is generally awful compared to steel blades (assuming iron armor), when you get into lower weapon metal tiers, they start to compare at least somewhat more favorably.

[Panando]

Speaking of axes, I saw your testing, and found the differences very interesting. In particular that axes did so poorly against a giant elephant, while they excelled in the tests I ran.

I suspect that it's not just the strength, but also the material of the axe when trying to effectively chop into a beast as large and tough as a giant elephant.

Yeah, a steel axe wielded by a very strong dwarf probably penetrates at least twice as deeply.

When I did my giant elephant testing we didn't have DFHack yet. Now with DFHack everything including quality can be tested.

I just tried a simple test with a Goblin with Grand Master combat stats with elevated-physical , vs an Animated Dead Giant Elephant with elevated-physical 100 (so it's very bad in combat and is just a big dumb punching bag.)

I used 4 variants of weapon: noquality iron axe, masterwork iron axe, noquality steel axe, masterwork steel axe.

Unsurprisingly the masterwork steel axe gets the fastest kill, but this is followed by the masterwork iron axe which takes about thrice as long to get the kill. And then the noquality axes appear to be simply incapable of killing the zombie giant elephant, eventually of course the goblin would pass out from exhaustion then get killed, but this took about 5x longer than it took the masterwork iron axe to get the kill. I only did a few trials, but this was pretty consistent, and as I understand this kind of combat it's not that random, there are no quick decisive lucky victories, the goblin just has to hit the Giant Elephant in the head enough times that the skull runs out of "hitpoints" and can get cloven, and the number of hacks was an average of 7 with the masterwork steel axe, 34 with the masterwork iron axe, and the other two axes got about 170 hacks to the head before the goblin got exhausted, so it wasn't like the other two axes were just unluckily not hitting the zombie in the head.

So something about the quality is enabling the masterwork iron axe to destroy the zombie whereas the noquality steel axe isn't just worse, it can't do it at all! My preliminary speculation would be that it has to do with whatever benefit blunt weapons get from being higher quality, and the edged->blunt conversion, because I'm pretty sure it can't just be about sharpness, since my understanding is a no quality steel axe should have about the same penetration as a masterwork iron axe. It's also obviously not about the to-hit bonus, at least not if the to-hit bonus is only about overcoming defensive skills, since the goblins are hitting every time regardless.

edit: Having run analysis on 8 combat reloads, here's the number of head hacks required for the goblin to defeat, or die to, the zombie giant elephant:

Code: [Select]

|         | Label         | Mean ± SD   | Head hacks                               |
|---------|---------------|-------------|------------------------------------------|
| Fight 1 | noq-iron-axe  | 165 ± 44.1  | [168, 193, 167, 58, 189, 181, 183, 180]  |
| Fight 2 | mas-iron-axe  | 34.1 ± 10   | [19, 45, 32, 26, 36, 46, 26, 43]         |
| Fight 3 | noq-steel-axe | 172 ± 18.2  | [165, 161, 196, 149, 188, 180, 186, 149] |
| Fight 4 | mas-steel-axe | 6.62 ± 3.16 | [5, 13, 4, 8, 4, 9, 5, 5]                |

So I only did 8 trials, but it can be seen that there's a dramatic difference in the number of head hacks required for a kill.

While this an extreme test, I think we can conclude that quality is exceedingly important to account for.

edit2: Since having combat logs to analyze is amazing, here is how I do it. The DFHack script "enable devel/annc-monitor" will dump reports to the DFhack terminal, but it's more useful putting them in a file. For this, the script/devel/annc-monitor.lua has to be modified, modify the contents of "function log(s, color)" inserting or replacing with the following lines:

Code: [Select]

    fd = io.open('reports.txt', 'a')
    fd:write(s, '\n')
    fd:close()
Thus writing announcements to a file "reports.txt" in the Dwarf Fortress folder.

[anewaname]

You should take a look at this thread, it showed there can be a bias in the arena based on unit list index, even when the teams were duplicates.

[Panando]

You should take a look at this thread, it showed there can be a bias in the arena based on unit list index, even when the teams were duplicates.

This is a really good point. I tried making a setup of 100, 1v1 fights between naked goblins armed with a Pick. Why a Pick? Because it has a single, high lethality attack, no pesky filler moves like slapping to introduce more noise.

I first painted Goblins 1-100 on Team 1
I then painted Goblins 101-200 on Team 4

This conflates unit# and team#, but it's common enough to do this I think.

I reloaded 20 times, and recorded the number of surviving goblins per team (does not sum to 100 because they can both bleed out, or potentially both survive but eliminated each other's ability to fight):

Code: [Select]

| Team 1 | 55 | 46 | 54 | 55 | 45 | 57 | 52 | 53 | 51 | 52 | 38 | 42 | 54 | 45 | 49 | 55 | 52 | 54 | 62 | 48 |
|--------|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|
| Team 4 | 32 | 44 | 34 | 35 | 45 | 32 | 39 | 41 | 41 | 35 | 49 | 44 | 37 | 45 | 41 | 34 | 40 | 38 | 32 | 40 |

Team 1: 50.95 ± 5.61 (SD), 95% CI [48.5, 53.4]
Team 4: 38.90 ± 4.97 (SD), 95% CI [36.7, 41.1]

While not enough trials to be absolutely conclusive, I would say this is strong evidence that unit order or unit team has a statistically significant effect on the result of closely matched fights in the current version of the game.

I then tried a second series of tests. In this case, I had 100 1v1 between unarmed goblins, instead of reloading I resurrected them using "full-heal", I did this 100 times since it's a lot quicker than reloading. Note that as resurrection brings back creatures unarmed, it's not easy to arm them. So this is punching, biting, kicking to death, meaning we expect less dramatic effects. (perhaps there are high lethality "glass cannon" creatures that would be better test subjects than goblins)

Team 1: 48.98 ± 5.74 (SD), 95% CI [47.85, 50.11]
Team 4: 46.61 ± 4.98 (SD), 95% CI [45.63, 47.59]

The 95% confidence intervals just barely do not overlap, again supporting the idea that team order or unit order is giving an advantage to team 1.

This suggests that data not biased by order is best obtained by having a target dummy team whose only job it is to die, and a striking team whose job is to strike down. Comparisons should only be made within a team, for example by measuring how long a target dummy takes to die to different weapons. This can compare both weapons by issuing different weapons to the striking team, and armor by issuing different armor to the target dummies.

[Blue_Dwarf]

What if you run the tests twice, reversing the teams, and averaging the results?

[Panando]

What if you run the tests twice, reversing the teams, and averaging the results?

It would get the average right but mess with the standard deviation.

[Salmeuk]

sometimes it feels like yall should be getting paid to run these tests.

the data as writ seems to suggest maces, spears, and hammers are rather niche in their effectiveness, at least when fighting on equal terms. the fact spears held the highest w/r against the colossus agrees with other posts I've seen

[Panando]

Incidentally I also want to add some interesting statistics from my combat logs of the tests vs the giant zombie elephants.

These fights involved a total of 14300 attack actions taken by the Goblins. These actions were broken down as follows:

Hacks: 69.9%
Punches: 8.6%
Strikes: 6.5%
Kicks: 4.6%
Scratches: 4.6%
Slaps = 4.3%
Bites = 1.4%
Pushes = 0.2%

So while it may seem that the game chooses attacks pretty randomly, at least in this instance the Goblins were choosing the best attack the vast majority of the time.

In terms of body part targeted:

Head: 28.9%
Body: 33.4%
Leg: 20.0%
Foot: 19.9%
Tusk: 2%
Neck: 0.7%
Tail: 0.6%
Ear: 0.2%

Nothing too surprising here, an Elephant has a pretty big head for a target.

These statistics do help to demonstrate that there isn't a ton of luck involved with the goblin choosing good attacks and good targets.

[Dwarf_Fever]

[snip]
Thus writing announcements to a file "reports.txt" in the Dwarf Fortress folder.

I didn't think about changing weapon quality in there, too, that's really great info - same about the logs.

[Schmaven]

Do martial trances happen in the arena?  I imagine a dwarf in a martial trance would beat a normal dwarf with any combination of weapons and armor.  It's possible some poor performance could be hidden by this.

[Panando]

Do martial trances happen in the arena?

Martial trances do occur within Arena. This is a reason some players use Goblins for testing instead of Dwarves. However if you set up the scenario as 1v1 fights they don't enter martial trances since it requires a dwarf to be fighting multiple opponents.

[Robsoie]

Shouldn't the warhammer/mace/whip be made of silver instead of steel to get their best damage or am i remembering wrong ?

edit : oh missed that part

Steel mace vs silver mace, using the above parameters. Steel mace won 59:41
Steel warhammer vs silver, ditto. It won with the exact same ratio of 59:41.

That's surprising though, as silver has a higher density than steel if i look at the values on the wiki, and up to know i was reading that higher density is always better in term of blunt impact ?
Has this changed in 50.x in comparison to past versions ?

[Blue_Dwarf]

Shouldn't the warhammer/mace/whip be made of silver instead of steel to get their best damage or am i remembering wrong ?

edit : oh missed that part

Steel mace vs silver mace, using the above parameters. Steel mace won 59:41
Steel warhammer vs silver, ditto. It won with the exact same ratio of 59:41.

That's surprising though, as silver has a higher density than steel if i look at the values on the wiki, and up to know i was reading that higher density is always better in term of blunt impact ?
Has this changed in 50.x in comparison to past versions ?

It hasn't changed, it's that many things you read about being "better" are somewhat exaggerated.