Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[j0nas]

There are many utilities out there for DF, I'm sure most players are familiar with Dwarf Therapist or one of the many similar utilities, for example.  They are currently based on pure memory manipulation, enterprising players sitting down to figure out what pieces of memory does what in a running copy of DF, then figuring out how to change that to make things easier to manage.  We could do a lot more than this, and do it much faster, simpler and with greater cross-version compatibility if there were some sort of API included in DF for doing this.  It doesn't have to be very complicated or fancy, a simple open TCP socket, by default only open for local connections, with a simple text protocol would probably be good enough for the entire thing.

Things that could be tamed to a much greater extent that way include, but are not limited to:
* Dwarf labors/names/mood
* Reading and designating in the stocks menu
* Adding and altering manager orders
* Mass designating preset build/dig orders, think pretty, OCD-friendly blueprints

This has probably been suggested many times in the past already, but this time is different.  This time is special, because...I added a note about this on the eternal suggestion voting list!  I have done my part, the rest is now up to you.  I need you, noble dwarf-herder in agreement, to head on over to:
http://www.bay12forums.com/smf/eternal_voting.php
..and search for the title "External API for DF utilities" and put down your vote for this.  (I recommend searching, because it's currently pretty far down in the list.)

[j0nas]

It seems that:
http://www.bay12forums.com/smf/eternal_voting.php#vote84
..might be a working direct link to the issue at hand on the list.  Your mileage may vary.

[Footkerchief]

^^^ Yeah, those only work if you've already voted for it.  I've been bugging zagibu to fix that.

Anyway, yes, this has been suggested before and Toady has made a couple direct responses.  Any supporters of the idea should definitely read these:

Spoiler: response (click to show/hide)

Somebody reported a post in this thread...  it didn't really matter which one.  A bit of chilling would be good.  There are a few people that have failed to chill.  There are quite a few statements here I'd respond to in person myself, and some very sharply, but internet discussions are more fragile and more prone to derailment.  So I'll try to stick to what seems to be the core of the matter.

Which for me (and granted, I'm picking a recent post -- I've read the thread, but don't remember when this came up first in this thread) is along the lines of what Jamuk said.  I'm being asked to assume risk.  I don't accept assessments of the risk as extraordinarily slight -- I've seen no sound basis for those assessments here.  I also don't think the currently proposed licensing/obfuscation method of putting the cat back into the bag would work -- with a principal downside being that it requires me to turn into an asshole who, after initially sanctioning the process, breaks everything that people have been working on.  Publishing an API and then trying to reel it back in just seems completely unworkable and irresponsible to me.  So the point is to assess risk upfront and then make a decision I can live with.

And right now, I have no idea what would happen.  This isn't the same as leaving my math job or adding the Z coordinate.  The first was a decision I had to make personally for my quality of life, and the second was a calculated risk which I didn't really have strong misgivings about (though some people are still pro-2D version, I think I can get most of them what they want over time in terms of gameplay difficulty, etc., though it'll never feel like it's quite on that rail again most likely).  Do I accept that third parties could improve the interface?  Of course.  Do I think I could do better than a third party myself, even over time?  No, not at all, though I can certainly improve on what I've got now to a large degree even without external input (which I, of course, love to receive).  So, initially, there's some attraction to the third party interface idea.

However, reflecting on the position I'd be in, there are things not to like about it.  How many threads were there about broken utilities when this version came out?  If more than half the player base comes in off a third party interface (and given how much the current interface sucks, and how much it is a source of first time downloaders dropping the game, this is not only imaginable, it is very, very likely), how would it be if it broke at each release?  There's no way to mitigate that without my direct involvement -- imagine a release down the line where you can suddenly move dwarven armies around on the world map, with a tactical view and various options.  That interface can't write itself, and it wouldn't be a quick patch, though certainly dedicated people, assuming that about whoever is maintaining the front-end at that time, can pull things together rapidly.  The pressure on me to work directly with them to get the interface out at the same time as the game itself would likely be immense and disruptive, given what little evidence we have from broken utilities.  That's not to say that I often get requests to work with utility writers (other than from the writers themselves, who I generally accommodate), but this would be at a different order of magnitude.

So what's the exact risk there then?  First, I don't want to work with other people.  So, assuming I don't do that, there's now a constant amount of pressure on me and a general disruption in the forums.  The latter could be mitigated with some announcements/guidelines, etc., but the pressure wouldn't go away.  Monetarily, it's impossible to say what on earth would happen, but if I supported the third party interface directly pre-release, I'd likely make more money, but I'd be unhappy.  If I didn't support it directly, but it was there, I'd still likely make more money, but I'd be unhappy.  That's not to say that I don't want to grow the audience and allow more people play the game, but I want to do it in a way with which I feel satisfied, even if that ends up being slower or just plain worse than a hypothetical third party alternative.  If someone, as a potential donor, thinks that's unreasonably selfish, that person shouldn't send me any more money than he or she thinks I've deserved for his or her enjoyment, just like everybody else.  Despite my dedication to this project, I'm unwilling to sacrifice my enjoyment of working on it for anything, including its quality or even its future release if it comes down to that.  That should be plainly obvious.  I'm not a slave.  Of course I do some things I don't like working on, with the interface and more.  I fulfill requests for features I'm not going to use myself.  However, again, the current proposal is of a different order of magnitude, with the potential of ruining it for me, and the end result is completely uncertain.

Given what I've seen here and there, it seems like a full third party interface might develop even without my involvement (rather than the various utilities we have now), but in that case, despite the same issues that might come up, at least it won't be a situation of my own creation that I feel a strong obligation to deal with, although the pressure would still be there.

Incidentally, in the proposed model, I still don't think I'd be able to bring code back in to the core project -- the first Kobold Quest port attempt showed that people bringing in code don't necessarily check or care about the licenses of the code they are swiping, and I can't assume that responsibility for them (it has stuff from Wine which shouldn't have been there).  Problems come up here with tilesets and so on too -- I know there've been a few cases where a contributor used somebody else's art and then it was put in a game, causing some legal trouble, though I don't have them at my immediate disposal.

I may have missed a lot, both problems I'm not thinking of and ways to handle them, since this gets complicated and is, due to lack of information, somewhat unspecified, but that's what I've got right now.

Spoiler: followup (click to show/hide)

I tried hard to put myself in your shoes and see things from your point of view, but apparently I failed to understand one thing (correct me if I'm wrong): the general atmosphere of the forums and feedback you receive from players is an integral part of your happiness. I was imagining that you could just let the mess settle down by itself after every release, but since I am not this forum's moderator, I can't really tell how it feels to have a big mess of threads about stuff not working.

It generally sucks to have a bunch of people complaining that things don't work, especially when there's something I can do about, that is, work with the interface modders more closely, having in this scenario sanctioned third party interfaces and released a public API.  It's the pressure that would make me unhappy.  It's good when the forum people are happy, but that wasn't what I was getting at so much.

When you speak of pressure, are you picturing direct pressure? I have a hard time imagining people screaming at you on the forums, except maybe for the occasional moron that can be ignored. Wouldn't people rather be putting pressure on the third-party developers to update their interface ASAP? But maybe you'd feel pressure nevertheless because you dislike seeing that people cannot play the game.

There would be direct pressure on me to work with third party developers -- this isn't about screaming.  This thread itself is direct pressure to work with third party developers (or at least get the ball rolling).  That isn't to say I disagree about you posting it.  I'm simply stating what it is, and I think that yes, there'd be a lot more of this, a lot more I'd have to reply to, and a lot of...  pressure.  Additionally, I'm not just talking about external pressure, in fact, principally not -- having created the situation, I would feel some obligation to make it run as smoothly as possible, and I would rapidly become irritated by this, because it's some of my least favorite kinds of work.

One thing I'd like to point out, though, is that you seem to think that a single interface, "The front-end", would come out of third-party development. Even though that's possible, open-source usually seems to bring diversity (look at the number of linux distros), especially in a single-player game where people don't have to use compatible clients, so they can play together. So I think we'd likely see an improved ascii client, a 2d tiles client, an isometric client, a 3d client, and maybe several of each one. In my view it would distribute the uproar, so that at least not everybody would be complaining about the same thing  . But I don't know if you see that as a relief.
[emphasis added]

He he, of course not.  It's worse actually -- more parties to work with.

Lastly, I may have a good way of mitigating the breakage of third-party clients at every release, without requiring your direct involvement. Assuming that you develop your own open-source interface at the same time as your closed-source server, you would likely publish your interface changes to a public SVN server. You would probably do it incrementally, at the rate of one or several commits a day - akin to backing up your work whenever you've made a step. In that case, you can bet that enthusiastic developers will watch those changes keenly, to see what's going on with the interface. This means that third-party client makers will already know that moving "dwarven armies around on the world map, with a tactical view and various options" is coming for the next version (from your very detailed dev notes), so they'll have time to plan which kind of interface they want to provide for that. And since they'll see the necessary code changes evolve gradually, because you'll always need an up-to-date interface for bugfixing, they'll be able to follow closely your development. They might have to adapt to your last minute changes, but otherwise I believe that the days of bugfixing prior to release could be enough for some of them to finalize their clients on time.
Now of course, you have to feel comfortable with that way of working.

Yeah, this somewhat mitigates breakage with those third parties that are active and approaching the situation with some dedication, but the trade-off of working with my day-to-day changes under that sort of scrutiny is definitely a minus for me, and I don't think it could really happen in a vacuum without questions about changes and so on, or without me feeling like I have to provide some sort of documentation as I add things.  I don't know if that sounds like quibbling, but I can't easily overstate how much I hate managing code or collaborating on code or anything along those lines.

Additionally, I think breakages would still happen, especially when staff leave projects hanging (work obligations, got tired of it, vacations, etc.), and it wouldn't change the fundamental situation I'm imagining, although in the case of project abandonment or delays with staffing, where there isn't pressure to work with the (missing) people in question, it's more the pressure to incorporate changes back into the base API, which as I stated in my previous long post, I wouldn't be able to do without potential legal problems.


I don't mean to write as if I'm sure about any of this, but once I drop this bomb in my project, I can see every day becoming "oh man, what a goddamn hassle" instead of what I've got now, which has less of that.  Again, I think the OP is justified in starting this thread, so I'm not raising that as an issue, but I've also spent some hours in here, reading posts and composing these replies.  That stuff just starts to magnify, and I'll either end up ignoring people I don't feel like I should be ignoring, or not ignoring people to the detriment of the core game itself and my own well-being.

What I would like to hear from Toady is what kind of graphics he has in mind for when he gets to the interface overhaul. That way we could at least prepare some sprites ahead of time and have nearly complete graphics sets as soon as the version with the new interface comes out. I personally waste a lot of my free time doodling in GIMP, so I might as well do something useful.

Depending on the stage of the interface overhaul, ultimately I'm going to be support 2D tilesets (probably in dimensions of multiples of 4 because I'm lazy with image file headers).  So if you want to draw up some 32x32s or something, you won't be wasting your time, I think.  It should be fairly straightforward to support single z-slice isometric stuff as well, once I get that going, since I'd just have to change the print locations and print order, though transparency decisions are probably annoying, and it's slightly more annoying to get multilayer isometric stuff going, since people are going to want various options about display there, so I don't really have a clear opinion on the future of isometric.  The support for a resizeable viewport/window is definite, but nothing has been decided on layering there (for instance, critters walking over grass tiles, that kind of thing -- people will want more and more out of this system, such as inventory and wound displays, and I'm not sure where lines will be drawn, or where it will bog down, anyway).

Also some earlier stuff:

Spoiler: part 1 (click to show/hide)

This has come up before, but I don't remember if I've actually entered into a discussion on it.  As it stands, without knowing much about it, my issues are as follows:

It seems like I'd have to publish all my class definitions to make this practical.  I don't anything about reverse engineering, but it doesn't seem far from opening up the source at that point.

The class definitions would also not document and otherwise support themselves, and there would be pressure on me to do so, which is not a one-time cost.

In fact, if a third party front-end becomes popular, I think I'd be forced into relationship or even a state of dependence on the third party that I'm not comfortable with.  And I don't think we can just gloss over version compatibility.  Ignoring it or handling it before releases -- either approach has ramifications for me.

Certainly players would benefit from having third party front-ends.  They'd benefit from open source and third party back-ends too.  If it means cutting myself out of my own game, I'm not going to support it.

I also don't know anything about scripting, and it's tough for me to say whether time spent learning a scripting language, implementing scripts and supporting third party front-ends is better spent adding modestly to the interface myself.

Spoiler: part 2 (click to show/hide)

"You break everything, we fix it, it's all good."

I don't think it's that simple.  Even the utilities that people use cause some grief when they are broken -- it would be far worse for a popular front-end.  If the majority of the player base has to wait after each release for third party fixes to a broken front-end that may or may not even happen, it puts a lot of pressure on me to resolve the issue by working more closely with the third party, a completely unknown entity that happens to maintain the front-end most people use.  It would be difficult for me to retain my independence once in that position.

I think somebody mentioned in an email discussion I had that in that case I should just incorporate some of the more popular aspects of third party interfaces back into the main release in order to avoid forming that kind of tie and to avoid serious bumps in playability for front-end users after each release, but I don't know the legal ramifications of that, and my first impression is that I'd be setting myself up for trouble, though some major publishers seem to do this without issue from what little I know about it.

I can vaguely see how something could be set up to automatically prepare a lot of this now, but it still seems like the functions etc. that are created would need to be documented to be useful, especially if the front-end needs to be patched quickly.  If variables are removed or changed around, it seems like the incompatibility for front-ends relying on a previous version of the automatically generated system could be pretty severe.

[j0nas]

Thanks foot, those are enlightening.  Unfortunately that means things don't bode well for an API of some sort, too bad.