Topic: DFHack: quickfort | buildingplan | blueprint | blueprints/library (Read 92176 times)

ldog · « **Reply #360 on:** September 01, 2021, 10:33:36 pm »

Quote from: myk on September 01, 2021, 10:30:26 pm

After some more testing, I've found that the "keep a small stock"-style manager orders do keep the larger one-time orders from ever getting scheduled in the workshops. This is bad, since it absolutely kills fort efficiency, but the fix is relatively simple. I added a "sort" command to the orders plugin. Running the command on repeat once a game day keeps things running smoothly. PR here
...aaaand it's merged! Thanks lethosor!

Yeah, I saw the PR, was gonna mention. It doesn't happen to me because I habitually send the 1 shot orders to the top of the queue.
Glad you made the change, we really shouldn't have to fool with it by hand.

myk · « **Reply #361 on:** September 03, 2021, 10:04:32 am »

Made a few final adjustments to the orders, combining "otherstock" into "basic" (there were only three items, and now that we have "orders sort", they're safe to put in basic without fear of holding up other orders).

I also updated the pre-built dreamfort upload at dffd: https://dffd.bay12games.com/file.php?id=15434
I should add a link to that download in the blueprint library docs.

ldog · « **Reply #362 on:** September 06, 2021, 01:50:55 pm »

So I see new release is out. Is there anything we were working on that didn't make it in?

myk · « **Reply #363 on:** September 06, 2021, 06:07:43 pm »

Many things, but there's always the next release.

I need to take a good slow read through the past few weeks of this thread, gather together the ideas, and work them into my backlog. Then I need to do a prioritization pass since there is always more work available than I have time to actually do :-)

I've been playing a long-running fort to make sure there aren't any emergent problems with our current setup. I embarked in a biome without any useful metals, getting all metals bit by bit through trading and sieges. I have to say I'm really pleased with the military manager orders you put together. They did an excellent job of automatically keeping my military equipped with the best weapons and armor that my fort could produce at any given time. Great job! I updated the dffd file (again) with this fort, which was a bit "cleaner" than my last uploaded fort. I'm also much happier with the minimal magma moving/pump 'n dump approach you described. Much less frustrating than giant pump stacks. I found it goes acceptably quickly once I have enough materials to make 12 magma-proof minecarts and 5 magma-proof wheelbarrows.

The prioritize script might need a little work. I need to do some more testing, but I think the prioritized jobs might be starving other job types that would otherwise get picked up. It's not feasible to just keep on adding prioritized job types -- there will always be the "next" one we need to add. One potential idea is to have prioritize periodically pause operations, like for a day or two per week. I'll need to play with it a bit to see if that's the right approach, though. The only jobs I saw getting ignored were "Cage animal" jobs, but I'd like to find a general solution that is robust for any playstyle.

The particular items on my mind for the immediate future are:
- blueprint repetitions and rotations for quickfort
- continuing with the blueprint plugin overhaul plan so generated blueprints are more useful and capture more map state
- preview (aka blueprint shadow) mode for quickfort
- special-purpose tools (e.g. prioritize, uniforms, linking levers, linking stockpiles, assigning minecarts)

ldog · « **Reply #364 on:** September 06, 2021, 08:03:23 pm »

Glad the military orders are working well, that was a smart test to see how they all interact with each other. Oh, adding "Flux stones not more than 5" on iron weapons/armor has worked well for me. If you have iron and flux then you have steel even if you aren't keeping up with production. Them making iron gear exacerbates the problem since even more iron needs to be made for your lagging steel production.

Did you find a way to get the magma-proof wheel-barrows to be used for the magma hauling?

I haven't really noticed any issues with the job prioritization running your default settings. If there's a lot of hauling to the depot that can cause a very short backlog but other than that I can't think of anything. It being somewhat time sensitive I can accept it.
The manager and tree cutting jobs aren't actually being created for whatever reason when they give me trouble, and those are my main problem areas lately.

myk · « **Reply #365 on:** September 07, 2021, 01:24:20 am »

Quote from: ldog on September 06, 2021, 08:03:23 pm

Did you find a way to get the magma-proof wheel-barrows to be used for the magma hauling?

it's fiddly, but no more than the rest of the "minimalist magma moving" procedure:

ensure all existing stockpiles have requested wheelbarrows fulfilled
make 5 steel wheelbarrows
forbid all non-steel wheelbarrows that are not assigned to a stockpile
set wheelbarrow count for the magma-filled minecart stockpile to 5
wait for steel wheelbarrows to get assigned to the stockpile
unforbid other wheelbarrows

Quote

I haven't really noticed any issues with the job prioritization running your default settings.

It may be nothing. Evidence is anecdotal at this point. I need to write a monitoring script that keeps time-to-completion stats for all jobs so I can do some proper analysis.

ldog · « **Reply #366 on:** September 07, 2021, 12:47:01 pm »

I guess that works. I just let them burn a few wooden ones, but then if I was in a treeless embark I'd probably be more inclined not to.

Oh, I updated the embark profile. The link should remain the same though, the (internal) title will have 2.0 in it.
Sanity check it for me if you get a chance.

Shell crafts could go into basic I guess (using fisher now that we've made it less obnoxious).

myk · « **Reply #367 on:** September 07, 2021, 04:59:29 pm »

Ok, what's next?

First, there are a few loose ends to tie up with Dreamfort and the supporting files, as well as bugs that have been getting on my nerves:

condition production of iron weapons/armor on the absence of flux stones (done)
look into misbehaving "Make gypsum plaster" order (filed df bug 11803)
add automation orders for sheets and scrolls (done)
add automation orders for shell crafts (done)
move coins from the goods stockpile to the metal stockpile (done)
move sheets from the goods stockpile to the textile stockpile (done)
move honey bee wax cake from the cookables stockpile to the goods stockpile (done)
implement "list" command for the orders plugin to see which JSON files are available (done)
fix automaterial plugin so that it designates construction tiles in a predictable order (the corner tile is always out of order, which is annoying when walling up light aquifers and need to do the corners first) (done)
fix workorder to not create milk/shear creature orders if similar orders already exist. this is especially a problem when you purchase a bunch of animals from merchants, the orders get generated, then you butcher the animals, leaving the orders "orphaned" (done - it actually already does this so no modification needed)

Then, I'd like to finish milestone 2 of the blueprint overhaul plan:

Support --splitby flag for "none" and "phase" strategies in blueprint and gui/blueprint (done)
Generate quickfort modeline labels from basename and phase
Support --format flag for "minimal" and "pretty" in blueprint and gui/blueprint
Support --playback_start flag in blueprint and gui/blueprint

Then I'll get back to automation tools:

implement "combine-drinks all" that iterates over all stockpiles
uniforms import/export
tool to create jobs that link levers to buildings with same name as lever
tool to assign free minecarts to minecart-less hauling routes
extend zone plugin to identify zones by name instead of id
autoassign animals of specific types/genders to specified pasture/cage

That's kind of a lot of work to queue up at once, so plans might change. I'd like to fit features for quickfort in there somewhere too, like preview mode and blueprint repetitions/translations. If there are requests from the community, I can bump those up in priority.

ldog: profile looks good. I like your decision to make the seventh dwarf a farmer instead of the craftsdwarf that I have. I think I'll borrow that. There is an old reference to making 4 nestboxes that I think no longer applies now that the profile only includes 2 geese.

ldog · « **Reply #368 on:** September 08, 2021, 08:43:21 am »

Thanks for checking! Yes, the farmer as a 7th worked out much better in the last few tests than the craftsdwarf or chef.
Without the farmer I felt like I just couldn't get caught up on the farming, even when the 1st wave brought a skilled farmer.

So I am crashing constantly with the new LNP, but the save is from the last one. Although it isn't like DF versions changed in between and I had been using the experimental DFHack anyway.
I'm not sure if it is to do with the magma hauling or something else. I've noticed the jobs being a bit fussy, like if a stockpile job got a hold of the cart it won't release, even if I force cancel it and assign it to a hauling route. Instead of queueing the job to put it on the route it requeues the stockpile job.

Such a strange fall it's been. Migrants on 1st day. A necromancer siege, a cyclops and then a wave of keas.

ldog · « **Reply #369 on:** September 09, 2021, 08:19:12 am »

I decided to add iron bolts since if we don't have copper we still need metal bolts (ran into that in that default world gen).
I made the order as a last resort rather than an upgrade, so the checks will prefer bronze>bismuth bronze>copper>iron.

I also got around to adding bismuth bronze weapons/armor, it sits between bronze and iron. It's a slight upgrade since it's a little more valuable, but more importantly it can stretch out the tin supply a little.
I left the bolts below normal bronze though.

https://drive.google.com/file/d/1fAu-lae1_HewP_DEX2hyR_dGNyUfRw18/view?usp=sharing
This is the add and changes (to bronze and copper), it also has the flux on iron. Does not have the unchanged military items.

Slurry (food, paste) should be removed from kitchen and moved to ~~textiles~~ misc liquids since it needs a barrel.

myk · « **Reply #370 on:** September 11, 2021, 02:42:02 am »

Sorry for the slow reply -- I've been working on the blueprint plugin, updating the core data structures and algorithm and doing performance testing to make sure it works on large embarks (results in the PR, but short version: after several rewrites, it's now twice as fast and uses half the memory).

I put up the changes I've done to dreamfort and the supporting files here. Thank you for the orders updates! I'll get those incorporated before I merge. I also need to do some more playtesting to double check that the recent changes all work as expected.

ldog · « **Reply #371 on:** September 11, 2021, 08:25:22 am »

You feel comfortable with only 1 architect to start (cutting from masons)? I guess with the prioritize running it might be doable but I think surface7 6 is going to be a big issue when the mechanic is already overstretched making mechanisms, building levers, linking levers, building traps and loading traps. Although I probably should prioritize adding another engineer by 2nd wave at the latest anyway. I think industry3 might bog down too. I'm due for a new test though so will give it a try (old save still crashing frequently even several seasons gone by).

I think 20 flux is too much on the iron gear, I went with 5 in case it was being traded for or some other obscure case that I might have overlooked, otherwise I would have went with 1 honestly. I really don't want them to bother if the capability is there for steel.

The soldier professions I removed siege & pump as well as architect; off-duty I only want them hauling, reloading traps, recovering wounded. I have mixed feelings on the pumping, since it's great strength-training. Before we started doing this thing with the professions I used to enable it for everyone actually.

myk · « **Reply #372 on:** September 11, 2021, 05:23:35 pm »

The 1 architect is experimental -- I haven't tested any of these changes yet (which is why the PR is WIP). It's just that I noticed that dwarf 1 (the generalist) normally ends up being a hauler, so why not give him the "in-profession" skill? I'm just looking for opportunities to minimize manual fiddling with labors. We'll see if it works out.

re: number of flux: my mistake -- I had put 20 since I misremembered that as the threshold for pig iron making and I didn't want to deadlock, but it's 5. Will fix.

Good points with the soldier professions. I'll make that change too.

edit: PR updated. I reordered the orders to match their priority, shifted the conditions on the bolts from bolts of other types of metals to bars (like what we do for the weapons and armor), fixed of by one errors that we've had in the conditions (that is, I switched a bunch of conditions from at most to less than), and added bismuth bronze maces, which were missing.

I also changed the thresholds of weapons to 10 of their source bar type, leaving armor at 20. That way if we run low on materials, at least we'll have a few weapons. Not sure if that is balanced correctly, though. Will have to test.

ldog · « **Reply #373 on:** September 12, 2021, 11:45:33 am »

I meant removing architecture from the masons effectively cuts you down to 1. The engineer/hauler always had it as a job.
So it didn't really make a difference on industry level (even with only 2 migrants, 1 of whom was a child...FML), of course running surface3 first & having to wall up aquifers delayed it anyway, but I forgot it is only the furnaces & forge that need architecture at that point anyway, so it's no hardship. The prioritize jobs really shines here. Actually keeping the masons focused at this point is better anyway so it seems like a good change.

I like the light aquifer water supply you did. It's easy and works nicely. Should add it to general quickfort blueprints. Making aquifers great again!

By the way, you don't need to wall the corners of aquifers if you don't dig them out. The water only spawns on the orthogonals not the diagonals. Although it does look much neater as a full square.

2nd migrant wave was more generous and besides assigning a 2nd engineer I designated 2 melee, and really there's little excuse to not designate at least 1 more engineer by then. I'd say a craftsdwarf, chef, laborer, 2nd farmer are a must. Everything else can wait until the 3rd wave if need be.

So remember I was saying platinum crossbows weren't used with "crossbow" as assigned weapon, well "individual choice, ranged" they will prefer the platinum. There is an iron, copper and several wood and they didn't grab any of them.

myk · « **Reply #374 on:** September 17, 2021, 03:44:40 pm »

That is a good point about not having to dig and wall up the corners of aquifers. At the start of the game, that would save a non-negligible amount of time and resources. Also yay for platinum crossbows : ) I'm glad they get used without manual poking.

Until I get them properly documented and merged, here are some of my "auxiliary" blueprints that I use:

extrasetup.csv: sets up a "clearcutting area" burrow for use with autochop, sets professions of starting dwarves with dwarf manipulator (assumes the example professions are defined and nothing extra is in the list), and disassembles the starting wagon

Spoiler (click to show/hide)

Code: [Select]

#aliases
# add "Clearcutting area" burrow for use with autochop
modburrows: ^wa&nClearcutting area&^^q
# set staring professions
setprofessions: ^ul{Tab 4}*p{Down 14}&{Down}x{Down}xp{Down 11}&A{Down}x{Down}xp{Down 8}&A{Down}p{Down 12}&{Down}p{Down 3}&^^q
#query
x{modburrows}{setprofessions}{F1}

aquifertap.csv: turns a light aquifer into a fresh water source for your wells

Spoiler (click to show/hide)

Code: [Select]

#dig
 , , , , , , , , , , , , , , , , , , , , , , , , ,#
 , , , , , , , , , , , ,r, , , , , , , , , , , , ,#
 , , , , , , , , , , , ,r, , , , , , , , , , , , ,#
 , , , , ,r,r,r,r,r,r,r,r,r,r,r,r,r,r,r, , , , , ,#
 , , , , , , , , , , , ,r, , , , , , , , , , , , ,#
 , , ,r, , , , , , , , ,r, , , , , , , , ,r, , , ,#
 , , ,r, , , , ,r,r,r,r,r,r,r,r,r, , , , ,r, , , ,#
 , , ,r, , , , , , , , ,r, , , , , , , , ,r, , , ,#
 , , ,r, , ,r, , , , , ,r, , , , , ,r, , ,r, , , ,#
 , , ,r, , ,r, , , , ,r,r,r, , , , ,r, , ,r, , , ,#
 , , ,r, , ,r, , , , , ,r, , , , , ,r, , ,r, , , ,#
 , , ,r, , ,r, , ,r, , ,r, , ,r, , ,r, , ,r, , , ,#
 ,r,r,r,r,r,r,r,r,r,r,r,j,r,r,r,r,r,r,r,r,r,r,r, ,#
 , , ,r, , ,r, , ,r, , ,r, , ,r, , ,r, , ,r, , , ,#
 , , ,r, , ,r, , , , , ,r, , , , , ,r, , ,r, , , ,#
 , , ,r, , ,r, , , , ,r,r,r, , , , ,r, , ,r, , , ,#
 , , ,r, , ,r, , , , , ,r, , , , , ,r, , ,r, , , ,#
 , , ,r, , , , , , , , ,r, , , , , , , , ,r, , , ,#
 , , ,r, , , , ,r,r,r,r,r,r,r,r,r, , , , ,r, , , ,#
 , , ,r, , , , , , , , ,r, , , , , , , , ,r, , , ,#
 , , , , , , , , , , , ,r, , , , , , , , , , , , ,#
 , , , , ,r,r,r,r,r,r,r,r,r,r,r,r,r,r,r, , , , , ,#
 , , , , , , , , , , , ,r, , , , , , , , , , , , ,#
 , , , , , , , , , , , ,r, , , , , , , , , , , , ,#
 , , , , , , , , , , , , , , , , , , , , , , , , ,#

and my personal favorite:
prisoner.csv: adds a prisoner quantum stockpile in the dreamfort barracks, taking from the animal stockpile 5 tiles to the north. I'm considering integrating this one into dreamfort since I find it so useful, despite the extra documentation that it will require (instructions for how to unforbid all, stripcaged all, ensure soldiers are training in barracks, assign stripped prisoners to pasture, etc.)

Spoiler (click to show/hide)

Code: [Select]

#aliases
permit_prisoner: {animalsprefix}{Right}s{search}&&^
prisoner_route_enable: {enableanimals}{cages}{permittraps}{permit_prisoner search=dwarves}{permit_prisoner search=elves}{permit_prisoner search=humans}{permit_prisoner search=giants}{permit_prisoner search=goblins}{permit_prisoner search=ettins}{permit_prisoner search=cyclopes}
#meta prisoner quantum dump
/2
/3
/4
/5
#place hidden()
,
c
#zone hidden()
,
n
#build hidden()
trackstopS
#query hidden()
{quantumstop name="Prisoner quantum" move={Up 5} move_back={Down 5} route_enable={prisoner_route_enable}}{givename name="prisoner dumper"}
{quantum name="prisoner quantum" quantum_enable={enableanimals}}{namezone name="prisoner processing"}

Also, status update: I've been working on the blueprint plugin and the gui/blueprint script. Updating the gui script to let you select a playback start position for the generated blueprint turned out to be much trickier than I thought it would be! GUIs are never simple...

I'm also writing a job latency analysis script so I can visualize how all our automation orders are interacting and see whether there are any more important job types that should be prioritized. I'm also curious what the totals are going to be for all the different job types. For every job type, I'll be tracking:

num jobs initiated
num jobs completed (along with how long each job took to be completed)
num jobs canceled

For measuring latency, I'm defining latency buckets that we can group by. I'm thinking we'll want to know:

is the job getting done faster than it needs to?
will a job get done before an item rots?
will a job get done before the player starts wondering "why isn't this done yet"?
will a job ever get done in a busy fort?

I figure this might translate to: 1 hour, 1 day, 1 week, and >= 1 month
Though I'll have to run some tests to see whether these latency cutoffs make sense. How long do different things take to rot?

News:

Author Topic: DFHack: quickfort | buildingplan | blueprint | blueprints/library (Read 92176 times)

ldog

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

myk

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

ldog

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

myk

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

ldog

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

myk

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

ldog

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

myk

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

ldog

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

ldog

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

myk

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

ldog

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

myk

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

ldog

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library

myk

Re: DFHack: quickfort | buildingplan | blueprint | blueprints/library