Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 2 3 [4] 5 6 ... 19

Author Topic: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors  (Read 178407 times)

Pirate Bob

  • Bay Watcher
  • [ETHIC: TORTURE_FOR_SCIENCE: ACCEPTABLE]
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #45 on: September 10, 2012, 02:29:21 pm »

I wonder what the outcome would be if you used modded metals? Given lead's density, it may be hellaciously damaging (it always seems to be in my games.) Platinum should have the same effect, but moreso. Aluminum, being light and stiff, would probably turn out like candy...

[/musing]
It is my suspicion that using lead/platinum won't make much difference, as at standard metal bolts already have enough momentum to pierce (or viciously bruise, if going through chainmail) vital organs.  I actually imagine standard metal bolts going right through the dorfs and out the other side in the absence of chainmail, so making them go further out the other side should have no effect :P.

Maybe it will somewhat reduce the (modest) protection dwarves get from wearing chainmail?  I will have to test it...

Nyxalinth

  • Bay Watcher
  • [LIKES_FIGHTING]
    • View Profile
    • My facebook page.
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #46 on: September 10, 2012, 05:12:47 pm »

awesome!  Do you study this stuff for school/use it for work?

I'm not mathy enough to do this sort of thing, but I respect anyone who can and does.

Great info to have.  So overall, it's better to have crap armor like leather or copper than none at all, but you want to replace crap ASAP, which was my near-downfall recently.  I had tons of copper weapons, and my dorfs barely dented invading gobbos. If it hadn't been for the caravan, I'd have been wiped out.
Logged
Nyxalinth likes the color blue, gaming, writing, art, cats for their aloofness,  Transformers for their sentience and ability to transform, and the Constructicons for their hard work and building skills. Whenever possible, she prefers to consume bacon cheeseburgers and pinot noir. She absolutely detests stupid people.

GreatWyrmGold

  • Bay Watcher
  • Sane, by the local standards.
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #47 on: September 10, 2012, 06:27:47 pm »

This, indeed, is ☼Science☼.
Of course, as an engineering major, I'm mostly interested in the practical applications, i.e. optimal bolt-making materials, which seem to be silver and steel (the latter being much more resource intensive).
Conclusion: Silver is the optimal material for bolts in terms of both availability and damage potential.1) (Fake) bone seems an adequate supplement, although the results are, of course, only approximate in that case.

In any case, I salute your impressive statistical analysis.
♪ so the ‼Science‼ gets done
and you make some neat *silver bolts [20]*
for the dorfs that are
Still Alive...♫

1) This seems to be a significant deviation from real-world physics, where iron and bronze are far superior bolt materials. Steel, of course, is quite effective in real life as well.
...Damn you, I just got that out of my head. Portal does have a lot of parallels to DF, though...
Logged
Sig
Are you a GM with players who haven't posted? TheDelinquent Players Help will have Bay12 give you an action!
[GreatWyrmGold] gets a little crown. May it forever be his mark of Cain; let no one argue pointless subjects with him lest they receive the same.

MaximumZero

  • Bay Watcher
  • Stare into the abyss.
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #48 on: September 10, 2012, 08:43:17 pm »

I wonder what the outcome would be if you used modded metals? Given lead's density, it may be hellaciously damaging (it always seems to be in my games.) Platinum should have the same effect, but moreso. Aluminum, being light and stiff, would probably turn out like candy...

[/musing]
It is my suspicion that using lead/platinum won't make much difference, as at standard metal bolts already have enough momentum to pierce (or viciously bruise, if going through chainmail) vital organs.  I actually imagine standard metal bolts going right through the dorfs and out the other side in the absence of chainmail, so making them go further out the other side should have no effect :P.

Maybe it will somewhat reduce the (modest) protection dwarves get from wearing chainmail?  I will have to test it...
I was more thinking about the ability to punch through breastplates.
Logged
  
Holy crap, why did I not start watching One Punch Man earlier? This is the best thing.
probably figured an autobiography wouldn't be interesting

Pirate Bob

  • Bay Watcher
  • [ETHIC: TORTURE_FOR_SCIENCE: ACCEPTABLE]
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #49 on: September 10, 2012, 08:53:49 pm »

I wonder what the outcome would be if you used modded metals? Given lead's density, it may be hellaciously damaging (it always seems to be in my games.) Platinum should have the same effect, but moreso. Aluminum, being light and stiff, would probably turn out like candy...

[/musing]
I ran platinum bolts vs. steel (helm, breastplate, chainmail, greaves, gauntlets, low boots), and found that it took only 14.4+/-0.4 hits to kill each dwarf and that all hits resulted in serious injuries (none just bruises).  So it appears your suggestion is correct, and they do in fact do significantly more damage.  They also don't seem to be slowed down much, if at all, by chainmail.  I'll do a full run soon...

Blakmane

  • Bay Watcher
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #50 on: September 10, 2012, 11:39:42 pm »

This is absolutely fantastic...

Out of interest, what are the variances/SEMs on your samples? A lot of people are talking about silver being better than iron/steel, but to me that doesn't look like it would achieve significance if you ran an ANOVA on it. Still an interesting result regardless: I had assumed silver would be a poor quality material but it is as good, if not better, than the big players.

Also, just an idea for automation: instead of setting your experiments up labouriously by hand, I wonder how easy it would be to create a macro which ran individual experimental sets on repeat, exporting the combat log in between runs. That way you could just leave the computer on and let it run for hours/days, coming back only to adjust the macro for the next condition. That'd vastly improve your sample size and reduce your workload.
Logged

MaximumZero

  • Bay Watcher
  • Stare into the abyss.
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #51 on: September 10, 2012, 11:51:10 pm »

I wonder what the outcome would be if you used modded metals? Given lead's density, it may be hellaciously damaging (it always seems to be in my games.) Platinum should have the same effect, but moreso. Aluminum, being light and stiff, would probably turn out like candy...

[/musing]
I ran platinum bolts vs. steel (helm, breastplate, chainmail, greaves, gauntlets, low boots), and found that it took only 14.4+/-0.4 hits to kill each dwarf and that all hits resulted in serious injuries (none just bruises).  So it appears your suggestion is correct, and they do in fact do significantly more damage.  They also don't seem to be slowed down much, if at all, by chainmail.  I'll do a full run soon...
Oooooh, thank you very much!

I wonder if you could do it with slade? :P
Logged
  
Holy crap, why did I not start watching One Punch Man earlier? This is the best thing.
probably figured an autobiography wouldn't be interesting

Emufarmers

  • Escaped Lunatic
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #52 on: September 11, 2012, 01:30:55 am »

I registered for these forums just to post how awesome this is.  Faith in humandwarfity restored, etc.
Logged

Zivilin

  • Bay Watcher
  • Dwarven Science Division
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #53 on: September 11, 2012, 05:50:52 am »

Hi everyone. Someone linked me to look at this study and I, having nothing better to do, typesetted it in LaTeX. Then someone suggested me to show this to the author, so here it is!.

(I also did the last one by OP) I like your style, keep it up!

Splendid! Now I just need to find a DF research paper to publish these in :P

awesome!  Do you study this stuff for school/use it for work?

What, crossbow performance analysis? That would be one fun school project/job! :P If You mean setting up objective experiments, defining useful criteria and performing very very basic statistical analysis, then yes, these are the things which go into standard college-level reports (except the statistical analysis, which should be much better then what I have)

I wonder what the outcome would be if you used modded metals? Given lead's density, it may be hellaciously damaging (it always seems to be in my games.) Platinum should have the same effect, but moreso. Aluminum, being light and stiff, would probably turn out like candy...

[/musing]
I ran platinum bolts vs. steel (helm, breastplate, chainmail, greaves, gauntlets, low boots), and found that it took only 14.4+/-0.4 hits to kill each dwarf and that all hits resulted in serious injuries (none just bruises).  So it appears your suggestion is correct, and they do in fact do significantly more damage.  They also don't seem to be slowed down much, if at all, by chainmail.  I'll do a full run soon...

That is interesting, because up until now bolt material had to have a significantly larger shear yield than armor material in order to cause tears: All bolts tore unarmored and leather, bronze/iron and higher tore copper, only steel and candy tore iron/bronze, and only candy tore steel. I suppose it is logical that weight also plays a role in this.

Out of interest, what are the variances/SEMs on your samples? A lot of people are talking about silver being better than iron/steel, but to me that doesn't look like it would achieve significance if you ran an ANOVA on it. Still an interesting result regardless: I had assumed silver would be a poor quality material but it is as good, if not better, than the big players.

Unfortunately, I opted for a very simple way of obtaining my means of arrows fired: counting all occurrences of "the flying" in the gamelog and then dividing it by the number of dwarfs which took part in the experiment. So I do not have direct access to standard deviation/SEM data, and I cannot perform ANOVA. I agree that silver-iron or silver-steel comparisons would probably result in ANOVA reporting borderline significance at best. That is why the conclusion of my study is "use any metal (except candy) against metal armor, and anything against unarmored opponents".

I can make a crude estimate on the SEM value of the Average of Fired Bolts (AofB). The AofB of most tests was around 13-18. I think it is safe to assume a standard deviation of no more than 25. This would mean that 95% of all bolts fall within <18-50,18+50> -> <0,68> (since You can't use up negative numbers of bolts). SEM = standard deviation divided by the square root of samples. There were 1000 samples per experiment. square root of 1000 = around 31. SEM = 25/31 = around 0.8. So an AofB derived from 1000 samples has a 95% chance to find itself within 1.6 of the true mean, assuming that the standard deviation of the samples is 25.

Hopefully I haven't made some obvious mistake along the way.

Also, just an idea for automation: instead of setting your experiments up labouriously by hand, I wonder how easy it would be to create a macro which ran individual experimental sets on repeat, exporting the combat log in between runs. That way you could just leave the computer on and let it run for hours/days, coming back only to adjust the macro for the next condition. That'd vastly improve your sample size and reduce your workload.

Both me and Pirate Bob did use macros. Mine is a fairly simple keyboard macro and I use it to set up a 1000 dwarfs at a time. Then I change their equipment to that of the defender and run it again. Each run is about 2 minutes. After everything is set I unpause the game and wait 5-10 minutes for the slaughter to end, then copy the gamelog to its own "Y vs X" text file, which I can then run data extraction scripts on. Mine is a crude setup, but it works fairly well. I was wondering how Pirate Bob's data gathering works - it seems on a much higher level.

I registered for these forums just to post how awesome this is.  Faith in humandwarfity restored, etc.

I am honored that our research could evoke such drastic action :)
Logged
No signature

Pirate Bob

  • Bay Watcher
  • [ETHIC: TORTURE_FOR_SCIENCE: ACCEPTABLE]
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #54 on: September 11, 2012, 07:29:56 am »

awesome!  Do you study this stuff for school/use it for work?
I recently earned a Ph.D. in physics, but I'm not sure how relevant that is to "this stuff" :P

I wonder what the outcome would be if you used modded metals? Given lead's density, it may be hellaciously damaging (it always seems to be in my games.) Platinum should have the same effect, but moreso. Aluminum, being light and stiff, would probably turn out like candy...

[/musing]
I ran platinum bolts vs. steel (helm, breastplate, chainmail, greaves, gauntlets, low boots), and found that it took only 14.4+/-0.4 hits to kill each dwarf and that all hits resulted in serious injuries (none just bruises).  So it appears your suggestion is correct, and they do in fact do significantly more damage.  They also don't seem to be slowed down much, if at all, by chainmail.  I'll do a full run soon...

That is interesting, because up until now bolt material had to have a significantly larger shear yield than armor material in order to cause tears: All bolts tore unarmored and leather, bronze/iron and higher tore copper, only steel and candy tore iron/bronze, and only candy tore steel. I suppose it is logical that weight also plays a role in this.
I was also surprised by this for exactly the reasons you stated.  It is also possible that platinum bolts aren't causing tears through chainmail, but rather always result in chipped/broken bones due to the high force.  I will investigate this further.

Out of interest, what are the variances/SEMs on your samples? A lot of people are talking about silver being better than iron/steel, but to me that doesn't look like it would achieve significance if you ran an ANOVA on it. Still an interesting result regardless: I had assumed silver would be a poor quality material but it is as good, if not better, than the big players.

Unfortunately, I opted for a very simple way of obtaining my means of arrows fired: counting all occurrences of "the flying" in the gamelog and then dividing it by the number of dwarfs which took part in the experiment. So I do not have direct access to standard deviation/SEM data, and I cannot perform ANOVA. I agree that silver-iron or silver-steel comparisons would probably result in ANOVA reporting borderline significance at best. That is why the conclusion of my study is "use any metal (except candy) against metal armor, and anything against unarmored opponents".

I can make a crude estimate on the SEM value of the Average of Fired Bolts (AofB). The AofB of most tests was around 13-18. I think it is safe to assume a standard deviation of no more than 25. This would mean that 95% of all bolts fall within <18-50,18+50> -> <0,68> (since You can't use up negative numbers of bolts). SEM = standard deviation divided by the square root of samples. There were 1000 samples per experiment. square root of 1000 = around 31. SEM = 25/31 = around 0.8. So an AofB derived from 1000 samples has a 95% chance to find itself within 1.6 of the true mean, assuming that the standard deviation of the samples is 25.

Hopefully I haven't made some obvious mistake along the way.
This is totally correct, and your estimate of a standard deviation of 25 is quite reasonable.  I record the number of hits on each dwarf individually and then from this calculate the standard deviation and then divide by the square root of the number of dwarves.  From this I get SEM of 0.5 to 0.8, depending on the individual experiment.  Your SEM might be a bit higher for the wood/bone bolts, as it tends to be proportional to the total number of hits.

One note - since you are counting "the flying", you are actually recording the number of hits on dwarves, not the number of bolts fired.  It is likely there are few misses in your setup, but these are not recorded in any way in the gamelogs. 

Also, just an idea for automation: instead of setting your experiments up labouriously by hand, I wonder how easy it would be to create a macro which ran individual experimental sets on repeat, exporting the combat log in between runs. That way you could just leave the computer on and let it run for hours/days, coming back only to adjust the macro for the next condition. That'd vastly improve your sample size and reduce your workload.
Both me and Pirate Bob did use macros. Mine is a fairly simple keyboard macro and I use it to set up a 1000 dwarfs at a time. Then I change their equipment to that of the defender and run it again. Each run is about 2 minutes. After everything is set I unpause the game and wait 5-10 minutes for the slaughter to end, then copy the gamelog to its own "Y vs X" text file, which I can then run data extraction scripts on. Mine is a crude setup, but it works fairly well. I was wondering how Pirate Bob's data gathering works - it seems on a much higher level.
The scripts and macros I use are posted on DFFD.  Anyone (and especially Zivilin) should feel free to use/modify them if they desire.  It sounds like Zivilin needs some Dwarven graduate students if he is setting up each experiment individually :P.

I also started with just keyboard macros within DF.  I assume that Zivilin also figured out that one needs to manually trim out most of the superfluous commands to get the macros to run quickly.  I also found I needed to delete most of the non-dwarf creatures from the raws to speed things up.  I then wrote a perl script which modifies the raws, copies gamelog.txt to another file named with the armor/ammo used, and then sends keypresses to DF to start the next macro run.  This allows me to run a complete set of armor vs. ammo combinations with no user input.  Unfortunately the keypress part is Linux specific, but I'm sure there's something equivalent for windows if someone wants to figure it out.

My analysis also uses a perl script, and makes heavy use of perl's regular expressions to parse the log files.  In summary, scans through each line of the log, and if it contains "flying", it checks the dwarf number in the same line, and increments the number of hits on that dwarf.  Then, when it finds a death/unconcious/falls over for that dwarf, it sets the appropriate entry to the number of hits on that dwarf so far, and locks that category (so that it only counts the first unconcious/falls over).  For armor effectiveness, looks for the word "through" in each line, and if it's there, then checks for which injuries resulted, and if there were deflections.  I can post a heavily commented version of the script later if people are interested in understanding exactly how the code works, but I don't have time right now.

I registered for these forums just to post how awesome this is.  Faith in humandwarfity restored, etc.
I am honored that our research could evoke such drastic action :)
I also am similarly honored if my small contribution to this work played any part in your decision.

MaximumZero

  • Bay Watcher
  • Stare into the abyss.
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #55 on: September 11, 2012, 08:21:22 pm »

Wow. You guys are freakin' awesome. Keep up the good work, noble ‼SCIENTISTS‼
Logged
  
Holy crap, why did I not start watching One Punch Man earlier? This is the best thing.
probably figured an autobiography wouldn't be interesting

Pirate Bob

  • Bay Watcher
  • [ETHIC: TORTURE_FOR_SCIENCE: ACCEPTABLE]
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #56 on: September 11, 2012, 10:36:41 pm »

I have added platinum bolts vs. all metal armors with and without chainmail to my first post on page 3 of this thread.  They are significantly more deadly that all other bolts, presumably due to their high density.  Thanks for suggesting this MaximumZero!  I am working on slade...

Oaktree

  • Bay Watcher
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #57 on: September 12, 2012, 01:40:03 am »

This thread is explaining why my dwarf casualties seem to be fairly low compared to other forts I read about.  I keep my military cooped up when the goblins bring archer squads - until after the traps and/or my marksdwarves chop them up a bit.  And then when I do attack them with melee troops I minimize their effectiveness by attacking from a short line-of-sight (they get 1-2 shots off max) and prefer to use well-trained squads with good shield use, dodge, and/or armor use skills.

And as mentioned in the starting posts of the thread probably the most efficient of crossbow fire is suppressive.  If melee troops go in and start coup de gracing unconscious or crippled targets the marksdwarves shift fire to others instead of spending 15-20 bolts on a single goblin.
Logged
Armorer McUrist cancels forge steel mailshirt, interrupted by minecart

Zivilin

  • Bay Watcher
  • Dwarven Science Division
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #58 on: September 12, 2012, 05:39:54 am »

I wonder what the outcome would be if you used modded metals? Given lead's density, it may be hellaciously damaging (it always seems to be in my games.) Platinum should have the same effect, but moreso. Aluminum, being light and stiff, would probably turn out like candy...
[/musing]
I ran platinum bolts vs. steel (helm, breastplate, chainmail, greaves, gauntlets, low boots), and found that it took only 14.4+/-0.4 hits to kill each dwarf and that all hits resulted in serious injuries (none just bruises).  So it appears your suggestion is correct, and they do in fact do significantly more damage.  They also don't seem to be slowed down much, if at all, by chainmail.  I'll do a full run soon...
That is interesting, because up until now bolt material had to have a significantly larger shear yield than armor material in order to cause tears: All bolts tore unarmored and leather, bronze/iron and higher tore copper, only steel and candy tore iron/bronze, and only candy tore steel. I suppose it is logical that weight also plays a role in this.
I was also surprised by this for exactly the reasons you stated.  It is also possible that platinum bolts aren't causing tears through chainmail, but rather always result in chipped/broken bones due to the high force.  I will investigate this further.

If You saw my chips/jams plots, almost all X vs Y combinations have an identical percentage of chips (just under 50%). The exceptions are when the materials are the same, giving rise to jams, and when there are a lot deflections. If You found chips were higher than 50%, this would be an interesting result.

However, I do not think this would contribute to a higher death rate. By now I think it is very apparent that a ranged weapon (and probably any weapon in DF) can only cause semi-immediate death (infections and the like are not included) by hitting a particular body part with sufficient force. Lethal hits for the crossbow are limited to the head (by brain tear and upper spine tear/bruising), throat (tearing it) and upper body (middle spine tearing bruising or heart tearing). The probability of killing a dwarf is virtually the same as the probability of hitting any of these spots with lethal force. We know that generally, bolts made of materials "better" than the armor do better, because they have access to throat and heart tearing, whilst bolts made of materials worse than the armor do not, and can only cause brain or upper/middle spine damage.

I think the question here would be: why does very heavy platinum require significantly less hits to kill a dwarf, on average? What damage type is increased? My current theory would be that the small percentage of fractures of skull (around 5% of all hits to the head for most bolts) is completely eliminated, turning it into lethal damage (brain/spine tear).

Another, similar idea would be to analyze why candy bolts have such an... average AofB, worse than "better bolts vs worse armor", but better than "worse bolts vs better armor". Candy bolts must be doing less damage to one of the critical regions for this to occur.

One note - since you are counting "the flying", you are actually recording the number of hits on dwarves, not the number of bolts fired.  It is likely there are few misses in your setup, but these are not recorded in any way in the gamelogs.

Indeed? Then I assume You refer to situations where a bolt is fired and doesn't land on the tile occupied by the target dwarf? Because i do have messages "The flying {MATERIAL bolt} misses Arena Dwarf XYZ" - I understand these would be instances of a dwarf dodging a bolt which 'hit' him, not instances of a bolt flying past a dwarf?

This will make measuring the accuracy of long-range hits quite a nightmare to obtain -_-'' Hopefully, a distance of two tiles used in my current setup produces a minimal amount of complete misses.

This thread is explaining why my dwarf casualties seem to be fairly low compared to other forts I read about.  I keep my military cooped up when the goblins bring archer squads - until after the traps and/or my marksdwarves chop them up a bit.  And then when I do attack them with melee troops I minimize their effectiveness by attacking from a short line-of-sight (they get 1-2 shots off max) and prefer to use well-trained squads with good shield use, dodge, and/or armor use skills.

And as mentioned in the starting posts of the thread probably the most efficient of crossbow fire is suppressive.  If melee troops go in and start coup de gracing unconscious or crippled targets the marksdwarves shift fire to others instead of spending 15-20 bolts on a single goblin.

This does seem like the best strategy to take. If armor offers so little protection against ranged combat, then minimizing the number of shots fired at You is the only way to minimize losses.

I remember once one of my military dwarfs was caught in a goblin ambush outside of the fort. They were all melee fighters and my valiant Swordsdwarf chopped 4 of them to pieces before they fled. ome time later, another military dwarf, similarly trained and equipped, was caught by a goblin ranged ambush. He didn't stand a chance and was down by the third projectile. The  melee/ranged discrepancy does appear to be rather large.
Logged
No signature

GreatWyrmGold

  • Bay Watcher
  • Sane, by the local standards.
    • View Profile
Re: Dwarven Research: A Comparison Study on the Effectiveness of Bolts vs Armors
« Reply #59 on: September 12, 2012, 06:38:07 am »

Historical note: Bows, cross and otherwise, were extremely deadly back before guns let you kill the bowman first.
Logged
Sig
Are you a GM with players who haven't posted? TheDelinquent Players Help will have Bay12 give you an action!
[GreatWyrmGold] gets a little crown. May it forever be his mark of Cain; let no one argue pointless subjects with him lest they receive the same.
Pages: 1 2 3 [4] 5 6 ... 19