Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Quietust]

is it better to check gref:getType() == df.general_ref_type.BUILDING_HOLDER or building = gref:getBuilding(); if building then?

If you know that you're looking for one specific reference type, it's better to check getType() because that just calls a function which immediately returns an integer, while getBuilding has to perform a binary search through the list of every building in your fortress to find the one to which it refers (which isn't too slow but is definitely slower than checking the reference type).

[lethosor]

On another note, the job from eventful.onJobCompleted kept being destroyed resulting in some strange stuff happening (mostly crashes ) as I kept trying to use it. The fix was making a clone at the very beginning (job = dfhack.job.cloneJobStruct(job)). So, I'm wondering: is this (job being destroyed after leaving the event handler) by design? The job parameter in eventful.onJobCompleted is already a copy, so it feels kind of strange to make yet another copy of it in order to use it. (But it may be actually is by design because of object ownership and stuff -- if that's the case, pretty please put a note about it in the docs)

Did you have to make a copy of the job to use it safely inside the event handler, or after the event handler returned? Relying on the latter is generally not safe (not limited to this case) because the copy of the job is destroyed after the handler returns, and usually results in a crash more often in cases where you know the object you're dealing with is about to be deleted (as is the case with jobs that have completed). I'm not opposed to adding a note about it in the docs, but it's not necessarily specific to this event. If you are accessing the job long after it has been destroyed, I might suggest only making a copy of the data you need, as opposed to everything that's part of the job - that way, you don't need to worry about freeing your copy of the job.

[Erendir]

On another note, the job from eventful.onJobCompleted kept being destroyed resulting in some strange stuff happening (mostly crashes ) as I kept trying to use it. The fix was making a clone at the very beginning (job = dfhack.job.cloneJobStruct(job)). So, I'm wondering: is this (job being destroyed after leaving the event handler) by design? The job parameter in eventful.onJobCompleted is already a copy, so it feels kind of strange to make yet another copy of it in order to use it. (But it may be actually is by design because of object ownership and stuff -- if that's the case, pretty please put a note about it in the docs)

Did you have to make a copy of the job to use it safely inside the event handler, or after the event handler returned? Relying on the latter is generally not safe (not limited to this case) because the copy of the job is destroyed after the handler returns, and usually results in a crash more often in cases where you know the object you're dealing with is about to be deleted (as is the case with jobs that have completed). I'm not opposed to adding a note about it in the docs, but it's not necessarily specific to this event.

It was after the event handler returned. Inside everything worked. I think the main source of my confusion is the fact that the job object passed to the event handler already is already a copy (and that I haven't programmed for systems without garbage collector for quite some time). I kind of assumed the original was destroyed but the copy will live on.

If you are accessing the job long after it has been destroyed, I might suggest only making a copy of the data you need, as opposed to everything that's part of the job - that way, you don't need to worry about freeing your copy of the job.

Does this mean I should free the job created using dfhack.job.cloneJobStruct(job) somehow? What about my case, where this job is linked into the world?

[lethosor]

It was after the event handler returned. Inside everything worked. I think the main source of my confusion is the fact that the job object passed to the event handler already is already a copy (and that I haven't programmed for systems without garbage collector for quite some time). I kind of assumed the original was destroyed but the copy will live on.

Yeah, the lack of a garbage collector is part of the issue here, as well as the fact that the conventional use-case is for event handlers to be the only users of the cloned job. EventManager can't know whether anything retained a reference to the job copy it made, so the only logical options are to free the job copy immediately after event handlers return, or to never free it (and leak memory), so it chooses the former.

Does this mean I should free the job created using dfhack.job.cloneJobStruct(job) somehow? What about my case, where this job is linked into the world?

I forgot that you were trying to do that (I thought your own code was doing things with the copy of the job, not DF). In this case, ignore my advice - you will need to create a new job (like with dfhack.job.cloneJobStruct()) and link it back into the world.

[Erendir]

On a somewhat related note. Jobs from work orders seem to be placed to the workshops once a day. I.e. the process is something like this:
 1. a work order is created.
 2. manager gets the job to manage, arrives at his office and validates/activates the work order. It gets the green checkmark in the list.
 3. on the next day (as soon as date changes), actual jobs are created in workshops

Any chance we can call the function doing the job assignments from Lua?

[lethosor]

Any chance we can call the function doing the job assignments from Lua?

The usual answer to this type of question is "no", unless the function is virtual (which doesn't appear to be the case) and there is actually a single function doing what you want (which isn't guaranteed).

I was about to recommend taking a look at the "prioritize" script from this PR, but on second glance, I don't think this would have the effect you're hoping for for manager-created jobs.

[myk]

Yeah, "prioritize" would help make sure the job gets done promptly once it is assigned to a workshop, but it wouldn't help getting the job to the workshop any faster.

[PatrikLundell]

Scripts can assign jobs to workshops (e.g. to cook with specific ingredients), but then you bypass the whole Management thingy with your own logic.

[Bumber]

Has any research been done on pathing flags? I see add_path_flags in units.xml and unit_path_flags in advmode.xml, but they're just int32's.

If you search for the byte sequence 93F833 in the disassembly, you should end up in a subfunction that seems to be calculating them from unit flags. A ghostly unit carrying items seems to return 0x93F833 for the flags. I'd research them myself, but I'm busy with stuff relating to tree chopping (such as fluid pathing, which just uses an immediate value 0x0 for the pathing flags.)

[Quietust]

Has any research been done on pathing flags? I see add_path_flags in units.xml and unit_path_flags in advmode.xml, but they're just int32's.

If you search for the byte sequence 93F833 in the disassembly, you should end up in a subfunction that seems to be calculating them from unit flags. A ghostly unit carrying items seems to return 0x93F833 for the flags. I'd research them myself, but I'm busy with stuff relating to tree chopping (such as fluid pathing, which just uses an immediate value 0x0 for the pathing flags.)

I've done some limited research against those path flags in version 0.28.181.40d, and it's quite likely that most of them will still be correct. I've been mainly posting my findings on the DFHack Discord server (in the "#reverse-engineering" channel), but this is what I was able to find:

Code: [Select]

* 0x00000001 - desperate (set when very hungry or thirsty)
* 0x00000002 - reckless (set when insane or when force-moving as an Adventurer)
* 0x00000004 - BuildingDestroyer1 (allow wooden doors)
* 0x00000008 - Lockpicker (allow doors not connected to levers), also set for BuildingDestroyer2
* 0x00000010 - stuck in impassable building
* 0x00000020 - allow unrevealed tiles
* 0x00000040 - can't open doors
* 0x00000080 - unknown, related to doors
* 0x00000100 - creature can learn
* 0x00000200 - unknown
* 0x00000400 - can jump off cliffs (when insane)
* 0x00000800 - can fly
* 0x00001000 - allow shallow water (1-5)
* 0x00002000 - allow deep water (6+)
* 0x00004000 - allow underwater (7 and 3+ water above)
* 0x00008000 - allow shallow magma (1-5)
* 0x00010000 - allow deep magma (6+)
* 0x00020000 - allow under magma (7 and 3+ magma above)
* 0x00040000 - already in water
* 0x00080000 - already in magma
* 0x00100000 - walk on normal land
* 0x00200000 - immobile_land
* 0x00400000 - dwarf, walk_tag equals zero
* 0x00800000 - unknown, tied to an unused "tile liquid" flag

[Bumber]

@Quietust

Thanks. For the pathing subfunction I'm looking at:

Code: [Select]

0x4: True causes a check for the material of a hatch to be in range 419d to 618d (both inclusive.) Still BuildingDestroyer1.
0x8: True skips a check for pet_passable on hatch. Still Lockpicker.
0x10: True skips a check for pet_passable on hatch. Still "stuck in impassible building".
0x40: True causes a check for any hatch. Still "can't open doors".
From what little of the path flag code I've looked at:
Can learn is still 0x100.
exit_vehicle1 and exit_vehicle2 from unit_flags3 result in path flag 0x400. Not sure if it shares that with "jump off cliffs" or if that's been moved.

[Bumber]

What's the process for mapping a DF global variable? I found a large global array used in (fluid) pathing. It starts 8 bytes after "agreement_next_id" (on Windows 64-bit, 0.47.05.)

Entries are structs of size 12 (not pointers,) which I'm calling path_tile for now. The struct contains the block x, y, and z (for use with block_index,) then the x%16 and y%16 tile offset, and finally an unknown/unused value (potentially used in other kinds of pathing.) All these are int16_t.

I'm guessing the array has 240000 entries total, since it adds entries at (base of array + 0x15F900), then flip-flops on which part is reading/writing each iteration. There aren't any checks done on the length of the array, it just keeps going until an iteration finishes where it didn't find a tile with a fluid and a path_cost less than world.next_path_cost.

Function I'm looking at is sub_140157F80. Here's where it puts the initial block x into the start of the array:

Code: [Select]

.text:000000014015804E mov     cs:word_1416B1990, ax

[Quietust]

What's the process for mapping a DF global variable? I found a large global array used in (fluid) pathing. It starts 8 bytes after "agreement_next_id" (on Windows 64-bit, 0.47.05.)

Entries are structs of size 12 (not pointers,) which I'm calling path_tile for now. The struct contains the block x, y, and z (for use with block_index,) then the x%16 and y%16 tile offset, and finally an unknown/unused value (potentially used in other kinds of pathing.) All these are int16_t.

I'm guessing the array has 240000 entries total, since it adds entries at (base of array + 0x15F900), then flip-flops on which part is reading/writing each iteration. There aren't any checks done on the length of the array, it just keeps going until an iteration finishes where it didn't find a tile with a fluid and a path_cost less than world.next_path_cost.

Function I'm looking at is sub_140157F80. Here's where it puts the initial block x into the start of the array:

Code: [Select]

.text:000000014015804E mov     cs:word_1416B1990, ax

The variable you're looking at is named "line", and it's mentioned in the Global Variable Table (which is marked by 0x12345678+0x12345678+0x87654321+0x87654321 and begins at address 0x1410b5040 on Win64).
From what I recall, that array is only used as a temporary buffer for pathfinding calculations, and none of its state ever persists between operations.

If you think there's value in adding it to structures, then we can certainly add it.

[Bumber]

The variable you're looking at is named "line", and it's mentioned in the Global Variable Table (which is marked by 0x12345678+0x12345678+0x87654321+0x87654321 and begins at address 0x1410b5040 on Win64).
From what I recall, that array is only used as a temporary buffer for pathfinding calculations, and none of its state ever persists between operations.

If you think there's value in adding it to structures, then we can certainly add it.

Adding it would probably be better than creating my own huge array for calculations. On the other hand, Maps::enableBlockUpdates might cause DF to handle everything itself.

Here's some almost properly formatted code of the fluid pathing function:

Spoiler (click to show/hide)

Code: [Select]

void updateTileFluidPathCosts(x, y, z)
{
auto mb = dfhack.maps.getTileBlock(x,y,z);
if (mb != NULL)
updateFluidPathCosts(mb, x%16, y%16);
}

void updateFluidPathCosts(mb, x_off, y_off)
{
if (next_path_cost >= 0x77359400) //2 billion
wipe_path_cost = true;
//for loop iterate map.map_blocks backwards
//zero out mb.path_cost 2D arrays //use memset?
next_path_cost = 1;
wipe_path_cost = false;
else if (pathfinder.wipe_path_cost = false)
next_path_cost += 0x1312D0; //1.25 million

int8_t flip = 0;
int8_t flop = 1;
int32_t sub_iter_max = 1;
int32_t valid_connections = 0;
path_tiles_start->blockX = mb->map_pos.x / 16;
path_tiles_start->blockY = mb->map_pos.y / 16;
path_tiles_start->blockZ = mb->map_pos.z;
path_tiles_start->tileOffsetX = x_off;
path_tiles_start->tileOffsetY = y_off;

do //main loop
{
path_tile *path_tiles_A = path_tiles_start + 0x15F900*flip;
path_tile *path_tiles_B = path_tiles_start + 0x15F900*flop;

for (int32_t sub_iter = 0; sub_iter < sub_iter_max; sub_iter++)
int16_t cur_blockX = path_tiles_A->blockX;
int16_t cur_blockY = path_tiles_A->blockY;
int16_t cur_blockZ = path_tiles_A->blockZ;
int16_t cur_tileOffsetX = path_tiles_A->tileOffsetX;
int16_t cur_tileOffsetY = path_tiles_A->tileOffsetY;
path_tiles_A += sizeof(path_tile);
map_block *cur_mb = world->map->block_index[cur_blockX][cur_blockY][cur_blockZ];
map_block *temp_mb = NULL;

if (cur_blockZ < mb->map_pos.z)
cur_mb->designation[cur_tileOffsetX][cur_tileOffsetY].liquid_static = false;

if (cur_tileOffsetX > 0) //check for west edge of block
if ( cur_mb->designation[cur_tileOffsetX-1][cur_tileOffsetY].flow_size > 0)
if (next_path_cost < cur_mb.path_cost[cur_tileOffsetX-1][cur_tileOffsetY])
cur_mb.path_cost[cur_tileOffsetX-1][cur_tileOffsetY] = next_path_cost;
path_tiles_B->blockX = cur_blockX;
path_tiles_B->blockY = cur_blockY;
path_tiles_B->blockZ = cur_blockZ;
path_tiles_B->tileOffsetX = cur_tileOffsetX-1;
path_tiles_B->tileOffsetY = cur_tileOffsetY;
path_tiles_B += sizeof(path_tile);
valid_connections++;
else if (cur_blockX > 0) //check for west edge of embark
temp_mb = world->map->block_index[cur_blockX-1][cur_blockY][cur_blockZ];
if ( temp_mb && temp_mb.designation[15][cur_tileOffsetY].flow_size > 0 )
if (next_path_cost < temp_mb.path_cost[15][cur_tileOffsetY])
path_tiles_B->blockX = cur_blockX-1;
path_tiles_B->blockY = cur_blockY;
path_tiles_B->blockZ = cur_blockZ;
path_tiles_B->tileOffsetX = 15;
path_tiles_B->tileOffsetY = cur_tileOffsetY;
path_tiles_B += sizeof(path_tile);
valid_connections++;

if (cur_tileOffsetX < 15) //check for east edge of block
if ( cur_mb->designation[cur_tileOffsetX+1][cur_tileOffsetY].flow_size > 0)
if (next_path_cost < cur_mb.path_cost[cur_tileOffsetX+1][cur_tileOffsetY])
cur_mb.path_cost[cur_tileOffsetX+1][cur_tileOffsetY] = next_path_cost
path_tiles_B->blockX = cur_blockX;
path_tiles_B->blockY = cur_blockY;
path_tiles_B->blockZ = cur_blockZ;
path_tiles_B->tileOffsetX = cur_tileOffsetX+1;
path_tiles_B->tileOffsetY = cur_tileOffsetY;
path_tiles_B += sizeof(path_tile);
valid_connections++;
else if (cur_blockX < world->map.x_count_block-1) //check for east edge of embark
temp_mb = world->map->block_index[cur_blockX+1][cur_blockY][cur_blockZ];
if ( temp_mb && temp_mb.designation[0][cur_tileOffsetY].flow_size > 0 )
if (next_path_cost < temp_mb.path_cost[0][cur_tileOffsetY])
path_tiles_B->blockX = cur_blockX+1;
path_tiles_B->blockY = cur_blockY;
path_tiles_B->blockZ = cur_blockZ;
path_tiles_B->tileOffsetX = 0;
path_tiles_B->tileOffsetY = cur_tileOffsetY;
path_tiles_B += sizeof(path_tile);
valid_connections++;

if (cur_tileOffsetY > 0) //check for north edge of block
if ( cur_mb->designation[cur_tileOffsetX][cur_tileOffsetY-1].flow_size > 0)
if (next_path_cost < cur_mb.path_cost[cur_tileOffsetX][cur_tileOffsetY-1])
cur_mb.path_cost[cur_tileOffsetX][cur_tileOffsetY-1] = next_path_cost;
path_tiles_B->blockX = cur_blockX;
path_tiles_B->blockY = cur_blockY;
path_tiles_B->blockZ = cur_blockZ;
path_tiles_B->tileOffsetX = cur_tileOffsetX;
path_tiles_B->tileOffsetY = cur_tileOffsetY-1;
path_tiles_B += sizeof(path_tile);
valid_connections++;
else if (cur_blockY > 0) //check for north edge of embark
temp_mb = world->map->block_index[cur_blockX][cur_blockY-1][cur_blockZ]
if ( temp_mb && temp_mb.designation[cur_tileOffsetX][15].flow_size > 0 )
if (next_path_cost < temp_mb.path_cost[cur_tileOffsetX][15])
path_tiles_B->blockX = cur_blockX;
path_tiles_B->blockY = cur_blockY-1;
path_tiles_B->blockZ = cur_blockZ;
path_tiles_B->tileOffsetX = cur_tileOffsetX;
path_tiles_B->tileOffsetY = 15;
path_tiles_B += sizeof(path_tile);
valid_connections++;

if (cur_tileOffsetY < 15) //check for south edge of block
if ( cur_mb->designation[cur_tileOffsetX][cur_tileOffsetY+1].flow_size > 0)
if (next_path_cost < cur_mb.path_cost[cur_tileOffsetX][cur_tileOffsetY+1])
cur_mb.path_cost[cur_tileOffsetX][cur_tileOffsetY+1] = next_path_cost;
path_tiles_B->blockX = cur_blockX;
path_tiles_B->blockY = cur_blockY;
path_tiles_B->blockZ = cur_blockZ;
path_tiles_B->tileOffsetX = cur_tileOffsetX;
path_tiles_B->tileOffsetY = cur_tileOffsetY+1;
path_tiles_B += sizeof(path_tile);
valid_connections++;
else if (cur_blockY < world->map.y_count_block-1) //check for south edge of embark
temp_mb = world->map->block_index[cur_blockX][cur_blockY+1][cur_blockZ];
if ( temp_mb && temp_mb.designation[cur_tileOffsetX][0].flow_size > 0 )
if (next_path_cost < temp_mb.path_cost[cur_tileOffsetX][0])
path_tiles_B->blockX = cur_blockX;
path_tiles_B->blockY = cur_blockY+1;
path_tiles_B->blockZ = cur_blockZ;
path_tiles_B->tileOffsetX = cur_tileOffsetX;
path_tiles_B->tileOffsetY = 0;
path_tiles_B += sizeof(path_tile);
valid_connections++;

if (cur_blockZ < world->map.z_count_block) //check for top of map
temp_mb = world->map->block_index[cur_blockX][cur_blockY][cur_blockZ+1]
if ( temp_mb && temp_mb.designation[cur_tileOffsetX][cur_tileOffsetY].flow_size > 0 )
if (next_path_cost < temp_mb.path_cost[cur_tileOffsetX][cur_tileOffsetY])
if ( canPathDown(temp_mb, cur_blockX, cur_blockY, true, true, true, false, 0x0) )
//bool canPathDown(map_block, x_offset, y_offset, bool check_passable_low, bool check_passable_flow_down,
//                 bool skip_floor_bar_grate, bool check_pet_passable_down, path_flags)
temp_mb.path_cost[cur_tileOffsetX][cur_tileOffsetY] = next_path_cost;
path_tiles_B->blockX = cur_blockX;
path_tiles_B->blockY = cur_blockY;
path_tiles_B->blockZ = cur_blockZ+1;
path_tiles_B->tileOffsetX = cur_tileOffsetX;
path_tiles_B->tileOffsetY = cur_tileOffsetY;
path_tiles_B += sizeof(path_tile);
valid_connections++;

if (cur_blockZ > 0) //check for bottom of map
temp_mb = world->map->block_index[cur_blockX][cur_blockY][cur_blockZ-1]
if ( temp_mb && temp_mb.designation[cur_tileOffsetX][cur_tileOffsetY].flow_size > 0 )
if (next_path_cost < temp_mb.path_cost[cur_tileOffsetX][cur_tileOffsetY])
if ( canPathDown(cur_mb, cur_blockX, cur_blockY, true, true, true, false, 0x0) )
temp_mb.path_cost[cur_tileOffsetX][cur_tileOffsetY] = next_path_cost;
path_tiles_B->blockX = cur_blockX;
path_tiles_B->blockY = cur_blockY;
path_tiles_B->blockZ = cur_blockZ-1;
path_tiles_B->tileOffsetX = cur_tileOffsetX;
path_tiles_B->tileOffsetY = cur_tileOffsetY;
path_tiles_B += sizeof(path_tile);
valid_connections++;
flip = 1-flip;
flop = 1-flop;
sub_iter_max = valid_connections;
}
while (valid_connections > 0);
}

At least I know how it works now, if nothing else.

[A_Curious_Cat]

Hi!  I’ve been having a few problems with autochop. 

The first problem is that it seems to designate way too many trees for felling.  On one of my previous forts, my stocks screen listed something like 25,000+ logs and my computer ground to a halt for at least five minutes when ever I cursored down to the line for logs.  I checked the autochop dashboard and it said that it was set to keep only a maximum of 100 logs on hand at a time.  Something is obviously going wrong here if autochop is ignoring it’s maximum limit.

The second problem is that, because there are so many trees being felled (and therefore so many logs to be hauled), my dwarves are spending most of their time endlessly hauling logs.  This makes it take an extraordinarily long time to get anything else done (to the point that I think the delays involved have probably been an important factor in the failure of my past several forts).

The third is that, with the trees being felled all over the map, I tend to have dwarves spread out all over the map gathering and hauling logs.  This makes it impossible to call everyone inside and shut the bridge before a large portion of my dwarves are slaughtered by some titan or werecreature.

Any help/advice would be very much appreciated.

Thanks.