Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2 3 ... 6

Author Topic: A (kludgey) workaround for the disappearing caravan bug  (Read 34985 times)

A_S00

  • Bay Watcher
    • View Profile
A (kludgey) workaround for the disappearing caravan bug
« on: July 08, 2016, 07:58:27 pm »

The Problem

There's currently a bug in the game, where you receive the announcement that a trade caravan has arrived, but the merchants never enter the map.  Their items appear on the stocks screen, and zooming to their location takes you to the map edge.  The merchants themselves don't appear on the Units screen.  Once this happens, unless you savescum to a point in time before the arrival of the phantom caravan, you probably never get a caravan from that civilization again, and if it's the dwarves, you also stop getting migrants, since the caravan never leaves your fortress to report back to the mountainhomes.

This is bad.

-----

The Solution

Fortunately, there's a way around it.  Unfortunately, it's a bit of a pain in the butt.  You will need:
  • dfhack
  • Python, Excel, Bash, or some other utility that lets you automate fiddling around with text
  • Some patience
Step 1:  Save your game, and back up your save

This method involves savescumming at least once.

Step 2:  Teleport every single unit onto your map at once

We want to teleport the merchants onto the map, so they can load properly and go home (or be killed, if they've already been bugged for long enough that they go insane).  We're going to do this via dfhack's teleport script, which teleports a unit to wherever your cursor is.  The syntax for this script is:
Code: [Select]
teleport -unit [unit ID #]
Unfortunately, we don't know the unit ID #'s of the merchants we're trying to rescue, and we can't check with teleport's -showunitid argument, because they're not on the map.  So instead we're going to teleport every single unit onto the map at once.  There's a dfhack script called multicmd that lets you run more than one command at once.  We want to build a dfhack query that looks like the following:
Code: [Select]
multicmd teleport -unit 1; teleport -unit 2; teleport -unit 3......; teleport -unit n
...where n is the highest unit ID # you're teleporting in.  In my game, n=4000 was enough, but I've heard ID's as high as the 13000's reported; it probably depends on how long you've been playing your particular fort and how many units have been generated in thiat time.

Obviously, typing out 4000+ instances of that command is going to be prohibitively tedious, so you're going to need to build this query in some automated way.  I did it with the following Python scriptlet:
Code: [Select]
toPrint = 'multicmd '
for i in range(1,4000): # Change these numbers to choose which ID's to teleport
    toPrint += ('teleport -unit '+str(i)+'; ')
print(toPrint)
This prints the query you want to run into the Python terminal; just copy+paste it into the dfhack terminal and you're good to go.

Fleeting Flames, in my help thread, suggests another solution using Excel:
Python is a great tool, but instead of learning it it is probably faster to use table calculation software such as excel or openoffice to generate 3 columns for "teleport text" <iterated unit id> "teleport text", select all, drag downwards with mouse, then copy all into notepad, run replace to remove the tabs and then save it and then tell dfhack to run it as script at fort startup or key command or what have you.

I'm sure the creative minds of these forums can come up with a few other clever ways to build the query.  If doing all the units at once crashes or hangs something, you might have to teleport in batches (i.e., teleport the units from 1-1000 in one query, then the ones from 1001-2000 in the next, and so on).  However you manage to do it, at the end of this step, you should have teleported every unit on the map to some easy-to-inspect location.

Note that this step will throw a lot of error messages in dfhack (one for each unit ID # you try to teleport that isn't actually a unit).  I just ignored them.

Step 3:  Find your merchants' unit ID #'s

So now you've teleported your merchants onto the map.  Great, that's what we're trying to do, right?  Well, yes, but you've also teleported every single citizen, Forgotten Beast, and Demon into the same spot, so you probably don't want to keep this save.  Our goal here is to find our merchants and figure out their unit ID #'s.

First, unpause for a few frames.  I found that once you teleported the merchants onto the map, they stayed as invisible phantoms for a few seconds of game time before properly loading in.

Once the merchants load in properly (probably !!on fire!!, bleeding from every orifice, and running from an Orthoclase Fluffy Wambler which Undulates Rhythmically), pause the game again, hover your cursor over the merchants and their pack animals, and query their unit ID #'s in dfhack with the following command:
Code: [Select]
teleport -showunitid
In my experience, all merchants from the same caravan will have ID's very close to one another (typically all within a range of about 100).  You can either record this whole range, or painstakingly record the unit ID # of each individual merchant.  I recommend the former.

Step 4:  Reload and teleport in only the merchants
  • Crash out of Dwarf Fortress without saving, and reload your game from before the teleport shenanigans.
  • Rebuild your dfhack query so that it only teleports the unit ID #'s (or small-ish range of ID's) that correspond to your phantom merchants.
  • Run the command.
  • Unpause and wait a few frames.
  • Your merchants should appear out of thin air at the location of your cursor.
Step 5:  Profit

In my game, it had been long enough that all of the merchants had gone insane.  A couple of them went berserk and my military killed them; the rest succumbed to melancholy and wandered slowly off the map.

However, the very next fall, a new Dwarven caravan, the outpost liaison, and a migrant wave appeared for the first time in several in-game years.

-----

Hope this is helpful to any of you who are stuck dealing with this bug.  Extra special thanks to Daris, without whose advice I would never have realized that the merchants could be teleported onto the map.

Let me know if you have any questions about anything in this post and I'll try to clarify.
« Last Edit: July 10, 2016, 11:26:46 am by A_S00 »
Logged

RocheLimit

  • Bay Watcher
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #1 on: July 09, 2016, 02:01:20 am »

This... sounds amazing if it works.  I will try it soon as I can tomorrow (later today?).  I've got a fort that hasn't seen a caravan from the dwarves or humans in decades & it would be awesome to order things from the mountain home once more/dump all the old clothes on them again.

*edit:  Unfortunately for my sanity, teleport is one of the DFhack features that has been FUBAR'd for my save (40.24).  Somehow, 3 or 4 different versions of plug-ins/scripts for DFHack got jumbled together and many do not work.  Yet the game runs... fine... and if I try to open it on a fresh install with updated dfhack versions, it won't get past the load screen.  I'll keep this method in mind, however, should I ever need it in another fort.
« Last Edit: July 10, 2016, 12:44:00 am by RocheLimit »
Logged

TheHossofMoss

  • Bay Watcher
  • "Man muss Heu machen, solange die Sonne scheint."
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #2 on: July 09, 2016, 03:37:48 pm »

Holy moly! I'm so thankful I haven't experienced this yet. I'm so sorry you are!

But dang, what a work-around! That's pretty awesome!
Logged
On the Fifth Day of Axemas, my love saved the fort from...
Five sieging Werebeasts, four Giant Dingoes, three sneaky Thieves, two drunken Black bears, and a Titan killing spree!

Kathe

  • Bay Watcher
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #3 on: July 10, 2016, 04:01:38 am »

I tried this because I have had invisible dwarven merchants on my map edge for the past 5 years, but I can't get it to work. If I -showunitid of something I can see, I can teleport it fine, but teleporting everything at once gives me the message "A unit must be specified with -showunitid blah blah". Any ideas on what I'm doing wrong?
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #4 on: July 10, 2016, 09:02:00 am »

What exact command do you use?

Kathe

  • Bay Watcher
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #5 on: July 10, 2016, 10:02:28 am »

First I tried
teleport -unit x
Where x is the -showunitid value I got from a nearby dwarf (7400 something). This command works fine.

When I try
multicmd teleport -unit 1; teleport -unit 2; etc
I get error messages.

I used Excel to make the list. I got it to 8200.
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #6 on: July 10, 2016, 10:12:05 am »

Did you replace the tabs excel inserts?

Also, use spoilered [CODE] tags - or pastebin, if too big for post - for the exact command you use. The reason for exact? There's probably a typo, possibly caused by line rollover or something in it. Or those tabs.

A_S00

  • Bay Watcher
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #7 on: July 10, 2016, 11:23:59 am »

I tried this because I have had invisible dwarven merchants on my map edge for the past 5 years, but I can't get it to work. If I -showunitid of something I can see, I can teleport it fine, but teleporting everything at once gives me the message "A unit must be specified with -showunitid blah blah". Any ideas on what I'm doing wrong?

This is the error message dfhack will give you every time you run the teleport command on an invalid unit ID.  Since you're running the command on every possible unit ID, a whole lot of them are going to be invalid; when I ran this, my dfhack terminal filled up with many screens' worth of that message.  However, it should keep trying to run the next command on the list, and the ones that are valid unit ID's should work fine.

Have you let it run the whole way through and checked whether it's teleporting stuff despite the errors?  If you want to do some proof-of-concept, you can try it on a smaller range (say the ID's from 1000-1500) before doing the whole range.

I'll edit the OP to be clearer about the fact that error messages are expected.
« Last Edit: July 10, 2016, 11:25:31 am by A_S00 »
Logged

Kathe

  • Bay Watcher
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #8 on: July 10, 2016, 12:52:58 pm »

Did you replace the tabs excel inserts?

Also, use spoilered [CODE] tags - or pastebin, if too big for post - for the exact command you use. The reason for exact? There's probably a typo, possibly caused by line rollover or something in it. Or those tabs.

This is what I pasted into dfhack, all the way to 8200 (without the []):
[multicmd teleport -unit 1; teleport -unit 2; teleport -unit 3; teleport -unit 4; teleport -unit 5; teleport -unit 6; teleport -unit 7; teleport -unit 8; teleport -unit 9; teleport -unit 10;]....

I did let it run through all the lines, it took about 40 seconds to finish. In the end, only 1 unit was teleported (after I unpaused the game), my mayor. Who is an elf.


What could I be doing wrong? Did I make some mistake in compiling the list? What tabs from excel do you mean? This is what I did:
1) Wrote [teleport -unit 1] in A1
2) Wrote [;] in B1
3) Wrote [ teleport -unit 2] in C1. I put a space before teleport so it spaces correctly for when I copy and paste into notepad and then dfhack.
4) Wrote [;] in D1
5) Selected C1 and D1 and dragged right up to 8200.
6) Selected the whole of line 1 and pasted into notepad, then from there copied and pasted into dfhack after writing multicmd.
Logged

Kathe

  • Bay Watcher
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #9 on: July 10, 2016, 01:04:34 pm »

Hmm wait, I think I got something wrong...I actually had to wait a much longer time (about 3 minutes) after the command had finished pasting itself into dfhack, and SOME units teleported during that time. Not all though, there are dwarves and animals on the map that haven't been teleported. Is the range 1-8200 wrong somehow? Could my fort have a range of 5000-unknown, instead of starting at 1?
Logged

Daris

  • Bay Watcher
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #10 on: July 10, 2016, 01:08:04 pm »

It's unlikely that it starts at 1.  Check one of your starting 7 and see what that dwarf's ID is (mine begin in the 6000 range).  Then check a unit that recently appeared (like a wild animal) and try teleporting in the units in between those numbers.

eta: although your outpost liaison might have a lower number, as the liaison may have been generated by the game prior to your starting 7.  The caravan merchants get incremental numbers, however, so if your liaison isn't stuck as well she should return with the next caravan once your current one is unstuck.
« Last Edit: July 10, 2016, 01:14:32 pm by Daris »
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #11 on: July 10, 2016, 01:17:02 pm »

When I replicate your list-compiling in openoffice, it does give all the instructions. The tabs are present, and even without removing them at least dozen-long list does work in linux - it gives the warning A unit needs to be selected or specified. Use teleport -showunitid to get a unit's ID. for each of the failed teleport, and then successfully teleports two units(mechanic and bookkeeper, here) whose ids I grabbed and told to teleport at the end of the list.

A_S00

  • Bay Watcher
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #12 on: July 10, 2016, 01:20:07 pm »

My valid ID's started around 1000, but it was a fairly young fort in a newly-generated region.
Logged

Kathe

  • Bay Watcher
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #13 on: July 10, 2016, 02:13:44 pm »

Thanks, it may be that I actually need to go wayyy further than 8200.
But the thing is, after I ordered the teleport on 1-8200, and not every unit on the map teleported, I checked those that hadn't. All of the ones I checked were in the range 5500-7500. So what caused them NOT to be teleported? Maybe giving it so many multicmds at once makes it skip some of the units. I'm going to try doing it in batches next A_S00 suggested, see what happens.
Logged

Miuramir

  • Bay Watcher
    • View Profile
Re: A (kludgey) workaround for the disappearing caravan bug
« Reply #14 on: July 12, 2016, 06:07:41 pm »

Thanks, it may be that I actually need to go wayyy further than 8200.
But the thing is, after I ordered the teleport on 1-8200, and not every unit on the map teleported, I checked those that hadn't. All of the ones I checked were in the range 5500-7500. So what caused them NOT to be teleported? Maybe giving it so many multicmds at once makes it skip some of the units. I'm going to try doing it in batches next A_S00 suggested, see what happens.

While I'm not familiar with the innards of dfhack, there are likely to be *some* kind of line-length limits and/or paste-length limits.  For instance, it might possibly be limited to 65535 characters (unsigned int), or some other seemingly-arbitrary limit imposed by the OS, libraries, coding shortcuts, or whatever.  Try a fairly small chunk first, and then work up. 
Logged
Pages: [1] 2 3 ... 6