Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Aedom]

Have you thought about just having it output the attack information every tick by reading unit.action (I think that's the one). It gets a little slow if you have more than a couple creatures fighting, but you won't get any missing or duplicated attack events that way. It's how I started looking at attack numbers to try and better understand the system. Although I was mainly trying to figure out what the values for the various things were. I made a quick script that just outputs the attack information. So I got an output like,

Code: [Select]

target_id item_id target_bp attack_bp attack_id unk_28 unk_2c unk_30 flags unk_38 unk_3c timer1 timer2 strength agility unit race caste
58 0 -1 7 -1 0 0 0 12 0 20 1 2 1000 1000 52 465 0
57 0 -1 5 -1 0 0 0 12 0 24 1 2 1000 1000 53 465 0
55 -1 8 9 0 2 -1 88 6 103 21 2 3 1000 1000 54 465 0
54 -1 15 9 0 0 -2 144 134 103 18 1 5 1000 1000 55 465 0
63 -1 2 9 0 2 0 97 6 103 26 2 3 1000 1000 56 465 0
53 -1 13 9 0 2 0 48 38 103 26 1 2 1000 1000 57 465 0
52 -1 11 9 0 0 0 78 262 103 24 4 3 1000 1000 58 465 0
52 -1 74 82 6 3 4 96 5 102 25 2 3 1000 1000 59 465 0
66 -1 11 9 0 2 0 84 6 103 19 2 3 1000 1000 60 465 0
65 -1 9 8 1 1 0 80 262 103 25 4 3 1000 1000 61 465 0
64 -1 0 8 1 1 0 97 6 103 26 2 3 1000 1000 62 465 0
56 -1 60 58 4 5 3 138 71 103 23 3 4 1000 1000 63 465 0
Since you have the target_id and target_bp available you can easily get wound information from the target as well.

Thanks for the fantastic suggestion.  I'll be able to play with the lua side of things either tonight or tomorrow.

If it ends up being significantly slower than usual, I don't see that as a problem.  Our use case is to be able to run arena mode and then export the data for external analysis.

This looks very promising!  Could post your lua script here in a spoiler tag, maybe?  I'd love to see it.

[Aedom]

I should also say that I've discovered cases where the attack event order does not agree with the combat log order.  This is very rare, but it can happen.  I have been working on the analyzer app, so that it can detect uncorrelated combat log message. 

In the end, I'm hoping Roses's solution saves me from having to rely on the mere detection of errors.

[Roses]

I'm traveling for the next couple of days so I don't have access to my DF stuff. But I can post my code on Wednesday or Thursday for you to look at.

[Aedom]

I'm basically finished with set of 1.0 features that I want, but before I show off the final teaser screenshots, I should report on my analysis regarding the event system accuracy.

The event system has been shown to be missing events, duplicating events, and sometimes giving the event out of order.  The heuristic system in the external analyzer app has been able to cope with this to the point that I can reasonably blame the apparent errors on the df hack lua eventful system.  It's a great tool, but unfortunately with its polling mechanics, I don't think it will be able to achieve 100% accuracy.  Repeat, I think that is OKAY for our uses.

I did experiment with turning down the calculation fps to 1, and it had no effect on the error rate.
I did experiment with pausing and stepping forward one frame at a time, it had no effect.
I did experiment with the unit.actions pattern that Roses suggested.  In the end, I decided that our accuracy level is fine without having to resort to taking live samples every tick to ascertain my own high level events.

That said, I just made the external analyzer app detect missing events and highlight them for you.  If we are going to be wrong, at least we can know we are wrong   Furthermore, if any great and altruistic person ever took it upon themselves to try to increase the accuracy of eventful, then our tool might be a helpful aide.

With out further ado, I will now show the design I finally went with. 

There are two parts to the solution. 

The first is a combat sniffer lua script that you drop in %DF%/hack/scripts.  To use the script, you go into arena mode and design whatever kind of battle you would like to record.  When ready to start, you run the sniffer in the df hack window.  This creates a "session", which is just a helpful way to group your combat data.  You can also optionally give the session a name.

The result of running the sniffer is a combat-sniffer-log.txt that gets dropped in your %DF% directory next to game log.

The second part of the solution is the external analyzer app.  With it, you can open sniffer log file, and it will show the results in exploded form for your analysis.  I will let the screenshots answer the big questions for me, and you guys can point anything confusing.  The ui design is basically just a 3 tab display, and here they are in order.

Report log display:

Spoiler (click to show/hide)

The Report Log tab highlights any combat log text that looks like combat information.  If we were able to map it to a discrete event, then it is highlighted green.  If we failed to find an event for the line, it is highlighted red.  Sometimes it is red just because I haven't written a black list rule for things like "the elephant gives into pain", but most of the time it is because of the lua event system dropping data.  My analysis of nearly 40 fights has shown ~95% accuracy, which I think is good enough.

The highlighted lines hyper link to the discrete strike data.

Strikes display:

Spoiler (click to show/hide)

This tab shows the stream of recognized attack strikes (no ranged yet), and you can see the resulting wound information, per body part tissue layer.

The attacker/defender lines hyper link to the units display.

Units display:

Spoiler (click to show/hide)

This tab lets you see all the body part information about the units involved, including body parts, tissue layer properties, attacks, armors, and weapons.


Anyway, thanks for anybody still hanging on to this thread.  I can't wait to get it out in beta test so I can see how you guys use/break it   I will report back with links and things once I'm ready for launch.

[Aedom]

I'm traveling for the next couple of days so I don't have access to my DF stuff. But I can post my code on Wednesday or Thursday for you to look at.

No need.  I settled on my level of accuracy vs amount of data trade off and am moving towards wrapping up 1.0.   I did check out unit.actions, which I did not know about.  Thanks for the pointer.

[Aedom]

Combat Analyzer v0.5 is out!

I created a separate thread regarding the tool itself.  That thread can be found here : http://www.bay12forums.com/smf/index.php?topic=160821.0.

Download : https://github.com/DigitalLibrarian/DfCombatSniffer/releases
Meager Instructions: https://github.com/DigitalLibrarian/DfCombatSniffer/wiki

The remainder of this thread should be focused on coming up with a mathematical description of how the combat effects are calculated.

[Aedom]

Here is something interesting that I noticed using the sniffer analyzer tool.  In the Strikes tab, if you click on a wound, there is a property called STRAIN.  I had originally thought these was just the stricken material's strain at yield, chosen from the appropriate material property set. This describes the situation most of the time, however, there cases were this value is clearly dynamic, having values that are fractions of the material strain at yield.

One outstanding example was a case where the strain for a part with torn apart skin was 50010.  The extra 10 shows that some type of calculation is going on.  Other times I have found the strain to be between 0-50000 (for a skin wound), indicating that the calculation is typically done as a fraction of the material strain at yield.

Currently working towards a formula to predict this strain value. 

If you read through the pre-existing literature regarding how momentum passes through the layers, you'll find that the soft tissues (with strain at yield >= 50k) all trigger a "blunt bypass" effect, thereby effectively skipping layers for momentum costs and what not.  If I understand it, it should not be possible to "dent the skin" under this model, because the "blunt bypass" would always be invoked first.  However, if you run enough arena battles, you will see blunt damage dent the skin. 

My hunch is that this blunt bypass feature is triggered off a dynamically calculated strain, rather than picking the value from the material raws, which opens up the possibility to dent the skin in some circumstances.

[Aedom]

Here is an interesting case.  Look at the two expanded strike results.  This is from a big free for all where I gave a bunch of dwarves silver maces.  The two highlighted are both the first strikes on an upper arm.  Dwarf 2 swings with a momentum of 136 and Dwarf 7 swings with 145.  However, the results are very different.  In both strikes, each of the tissue results were the same so I only highlighted one in each strike result. 

The big question is:  If Dwarf 7 had more momentum, then how come Dwarf 2's swing managed to bruise the muscle, but Dwarf 7's swing only bruised the fat?

Spoiler (click to show/hide)

Spoiler (click to show/hide)

Clearly there is at least one more factor at play here.  They both did the exact same computed damage to all layers involved.  Perhaps there is an RNG roll at each layer that might stop the attack, or something...

[Roses]

Well there are random factors that get applied on each swing, effecting not just velocity and momentum, but the "squareness" of the strike (i.e. if it hits with full force or partial force). In addition there appears to be a sort of damage range that is selected from randomly on top of all of the other factors. Another difference could be the layer volume, was one dwarf fatter than the other? Did they have different attribute values? (i.e. STRENGTH, TOUGHNESS, etc...). Were there exhaustion levels the same? There are also different types of attack (i.e. wild, precise, strong, etc...). Some of these attacks have the same momentum modifiers but do different levels of damage.

From my limited testing I have had a single dwarf make the same attack against a dozen different dwarf clones (same attributes, same fat, same sizes, same everything) and have gotten slightly different results for each attack. Sometimes it's just a small difference with slightly different numbers, but sometimes it can be the difference between damaging different layers, all the way to cutting off an arm versus just cutting down to the muscle. The variability can be astonishing. That's why I started to use code that tracked thousands of attacks and performed statistical analysis on them.

Here is my simple code that tracks attacks

Code: [Select]

dir = dfhack.getDFPath()
io.output(dir..'/test.txt')
io.write('target_id,item_id,target_bp,attack_bp,attack_id,unk_28,unk_2c,unk_30,flags,unk_38,unk_3c,timer1,timer2,unk_4-wrestle_type,unk_4-4,unk_4-6,unk_4-8,unk_4-c,unk_4-10,unk_4-14,strength,agility,unit,race,caste\n')
output = {}
function callback()
 n = #output
 for _,unit in pairs(df.global.world.units.active) do
  output[n] = {}
  for _,action in pairs(unit.actions) do
   if action.type == 1 then
    table.insert(output[n],action.data.attack.target_unit_id)
    table.insert(output[n],action.data.attack.attack_item_id)
    table.insert(output[n],action.data.attack.target_body_part_id)
    table.insert(output[n],action.data.attack.attack_body_part_id)
    table.insert(output[n],action.data.attack.attack_id)
    table.insert(output[n],action.data.attack.unk_28)
    table.insert(output[n],action.data.attack.unk_2c)
    table.insert(output[n],action.data.attack.unk_30)
    table.insert(output[n],action.data.attack.flags)
    table.insert(output[n],action.data.attack.unk_38)
    table.insert(output[n],action.data.attack.unk_3c)
    table.insert(output[n],action.data.attack.timer1)
    table.insert(output[n],action.data.attack.timer2)
    table.insert(output[n],action.data.attack.unk_4.wrestle_type)
    table.insert(output[n],action.data.attack.unk_4.unk_4)
    table.insert(output[n],action.data.attack.unk_4.unk_6)
    table.insert(output[n],action.data.attack.unk_4.unk_8)
    table.insert(output[n],action.data.attack.unk_4.unk_c)
    table.insert(output[n],action.data.attack.unk_4.unk_10)
    table.insert(output[n],action.data.attack.unk_4.unk_14)
    table.insert(output[n],unit.body.physical_attrs.STRENGTH.value)
    table.insert(output[n],unit.body.physical_attrs.AGILITY.value)
    table.insert(output[n],unit.id)
    table.insert(output[n],unit.race)
    table.insert(output[n],unit.caste)
    table.insert(output[n],'\n')
    action.type = -1
    io.write(table.concat(output[n],','))
    n = n + 1
   end
  end
 end
 if n < 10000 then
  dfhack.timeout(1,'ticks',callback)
 else
  print('Done')
  io.close()
 end
end

dfhack.timeout(1,'ticks',callback)

[Aedom]

Well there are random factors that get applied on each swing, effecting not just velocity and momentum, but the "squareness" of the strike (i.e. if it hits with full force or partial force). In addition there appears to be a sort of damage range that is selected from randomly on top of all of the other factors. Another difference could be the layer volume, was one dwarf fatter than the other? Did they have different attribute values? (i.e. STRENGTH, TOUGHNESS, etc...). Were there exhaustion levels the same? There are also different types of attack (i.e. wild, precise, strong, etc...). Some of these attacks have the same momentum modifiers but do different levels of damage.

From my limited testing I have had a single dwarf make the same attack against a dozen different dwarf clones (same attributes, same fat, same sizes, same everything) and have gotten slightly different results for each attack. Sometimes it's just a small difference with slightly different numbers, but sometimes it can be the difference between damaging different layers, all the way to cutting off an arm versus just cutting down to the muscle. The variability can be astonishing. That's why I started to use code that tracked thousands of attacks and performed statistical analysis on them.

Here is my simple code that tracks attacks

Code: [Select]

dir = dfhack.getDFPath()
io.output(dir..'/test.txt')
io.write('target_id,item_id,target_bp,attack_bp,attack_id,unk_28,unk_2c,unk_30,flags,unk_38,unk_3c,timer1,timer2,unk_4-wrestle_type,unk_4-4,unk_4-6,unk_4-8,unk_4-c,unk_4-10,unk_4-14,strength,agility,unit,race,caste\n')
output = {}
function callback()
 n = #output
 for _,unit in pairs(df.global.world.units.active) do
  output[n] = {}
  for _,action in pairs(unit.actions) do
   if action.type == 1 then
    table.insert(output[n],action.data.attack.target_unit_id)
    table.insert(output[n],action.data.attack.attack_item_id)
    table.insert(output[n],action.data.attack.target_body_part_id)
    table.insert(output[n],action.data.attack.attack_body_part_id)
    table.insert(output[n],action.data.attack.attack_id)
    table.insert(output[n],action.data.attack.unk_28)
    table.insert(output[n],action.data.attack.unk_2c)
    table.insert(output[n],action.data.attack.unk_30)
    table.insert(output[n],action.data.attack.flags)
    table.insert(output[n],action.data.attack.unk_38)
    table.insert(output[n],action.data.attack.unk_3c)
    table.insert(output[n],action.data.attack.timer1)
    table.insert(output[n],action.data.attack.timer2)
    table.insert(output[n],action.data.attack.unk_4.wrestle_type)
    table.insert(output[n],action.data.attack.unk_4.unk_4)
    table.insert(output[n],action.data.attack.unk_4.unk_6)
    table.insert(output[n],action.data.attack.unk_4.unk_8)
    table.insert(output[n],action.data.attack.unk_4.unk_c)
    table.insert(output[n],action.data.attack.unk_4.unk_10)
    table.insert(output[n],action.data.attack.unk_4.unk_14)
    table.insert(output[n],unit.body.physical_attrs.STRENGTH.value)
    table.insert(output[n],unit.body.physical_attrs.AGILITY.value)
    table.insert(output[n],unit.id)
    table.insert(output[n],unit.race)
    table.insert(output[n],unit.caste)
    table.insert(output[n],'\n')
    action.type = -1
    io.write(table.concat(output[n],','))
    n = n + 1
   end
  end
 end
 if n < 10000 then
  dfhack.timeout(1,'ticks',callback)
 else
  print('Done')
  io.close()
 end
end

dfhack.timeout(1,'ticks',callback)

Thanks for those tips on run time variations.  That is exactly the type of stuff that I'm trying to chart out.

I've got a simulator that I use to test these ideas out.  It can read all the df raws and create prototypical creatures, but it doesn't do any of the creature instance-level variation.  All members of a creature species are identical in my sim.  I am capable of creating the lower bounds of strike results that I see if the combat analyzer.  I think I'm going to have to move to a statistical analysis model as you did, in order to make significant progress.

[Aedom]

Here's another thing I'm stuck on.  I have been focusing on understanding naked dwarves fighting with wooden swords.  I am able to predict most of the behavior I see, except that the combat analysis tool shows me a very surprising value for the sharpness of a wagon wood short sword.

Spoiler (click to show/hide)

It has a value of 500.  However, the WOOD_TEMPLATE has [MAX_EDGE:1000].  I don't understand how it gets this value.  My best guess is that the sharpness might be multiplied by a quality factory, and the arena creates average quality items with this factor = 0.5. 

That's a conjecture.  Does anybody know for sure how this 500 is arrived at as the sharpness for a wagon wood short sword?

[Roses]

I think I do remember seeing somewhere that quality affects sharpness.

So I never did ask, what is your end goal for this project?

[Aedom]

I think I do remember seeing somewhere that quality affects sharpness.

So I never did ask, what is your end goal for this project?

Oh I'm just having fun and practicing.  I have a parser for the raws and a time simulation that I think is comparable to what df can do.  No promises of anything ultimately coming out of my simulation experiment, but that isn't the whole project.

Along the way, I hope to make some tools to enable the modding effort and also open the model of how the maths operate.  Once this exploration thread has run its course, I was going to see about updating the df wiki with accurate combat model and statistics as another community aid.

One thing that I am going to release in the coming months is an open source df raws parser, that deserializes to .net pocos, suitable for linq queries.  I'm still feeling out the api I want to expose, but I have enough functionality to drive interesting applications already.  My sim is the current test bed for that.

[Roses]

Interesting. The reason I asked is because I have always wanted a way to simulate in game combat properly. This would allow us to write scripts in DFHack that would "send" units out on missions and such and have them come back with appropriate wounds.

[Aedom]

Interesting. The reason I asked is because I have always wanted a way to simulate in game combat properly. This would allow us to write scripts in DFHack that would "send" units out on missions and such and have them come back with appropriate wounds.

My sim is written in .net and is a standalone game engine.  Don't think I can help you there.  However, the model that we come up with could theoretically enable that.

I bet you could get pretty far by just using random wounds, maybe with probability assigned based on combat skills.  Players loves to infer meaning where there isn't any.