Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Andir]

You're confusing the interface with something that would have direct access to the game's logic. A third party interface would have no more capability than the official interface. Do you currently have the ability to delete a merchant in the interface as it stands now? Of course not. The interface would have two roles. Sending requests and outputting the current state of the game.

Let's start with sending requests. Say the player wants to build a construction at a specific location. Say a wall. Well the player clicks/types whatever has to be to select a wall and the location for it. The interface then sends a request to the game that a wall be placed at that location. The game then responds whether or not such a request was successful. Thus you've either placed a wall or the game responds back that it couldn't be done and the interface either refuses to place it or tells you that the attempt failed.

Now for output. The interface has decided it is time to render a portion of the game to the screen. The interface sends a request to the game. This request would amount to returning information on a visible tile/object at a certain location. The game would comply sending the data for that location. If the interface is set to render at a higher rate than the game currently updates then the interface could choose to cache the results it gets and keep using them over and over until a new game cycle occurs at which point the cached copies are labeled "dirty" and a fresh copy has to be requested.

As for threading, well threading the graphics wouldn't be too difficult at all. Simply have a thread that sits around doing nothing until receiving a command from the interface to render the current contents of the cache to the screen. The thread then immediately springs into action trying to render as fast as it possibly can. With hardware acceleration such a render shouldn't take very long at all. In fact in most cases the interface should be able to render fast than the game updates.

When it comes to threading I honestly have about as much experience as Toady does (though I do have some knowledge I picked up at college it isn't anywhere near as useful as experience). Perhaps even less. Thus I would probably avoid threading more than what can be easily threaded (like video, audio, events, etc). And you are right, syncing stuff can be slow. However the thread I mentioned above for graphics wouldn't have need of syncing beyond detecting if the cache has become "dirty" and possibly halting the current render if so.

Edit: I should mention that shortly after my last programming course ended (Java and it was last since I had run out of funds) I had started development of a text adventure game engine that involved a server/client like system using an API to communicate. It basically consisted of two commands. One fetched output and the other sent input. Not anywhere near as flexible as the interface for DF would have to be, but at least I can say I've tried it myself.

As a full time programmer who does multithreaded SQL injection and eLearning courses, interactions and material: I have to say you pretty much nailed it.

No synchronizing threads would need to be done.  The world would likely sit off in a class of it's own, the rest of the game would send a request to this "world object" for access to specific tiles.  Upon getting the request, a check is made to determine if those tiles are in use, stalling the waiting process for a few ticks for the other process to release.  If they are not in use, the process is a minor addition to the current methodology's CPU usage and thousands of times more scalable.  Worst case scenario, you go to build a workshop and a few dwarfs have to move out of the spot you are trying to place the shop before the "shop placing thread" has access to specify the build order.  If any other dwarfs are trying to gain access to those same tiles, they wait in line like everything else.  In a way, this emulates/replaces the current collision detection methods that slow dwarfs down now when they are all trying to cram down a narrow hallway.

Edit: I should clarify.  It wouldn't replace collision detection.  It would give a similar "hiccup" to the movement of dwarfs colliding like they do now.

[Blacken]

You'd only free up maybe 10ms of CPU time (read the Kobold Quest source code, it has DF's rendering method), which on most computers is at most 1 more frame. Also, you'd probably lose 20-50ms just by sychronizing the data accesses.

Huh, I didn't realize the DF renderer was that tight. OK, there wouldn't even be that much of a benefit.

Frobozz:
Yeah if they went the route of sockets. If they went the route of an API like I suggested there wouldn't be a loss due to syncing things. The amount of overhead would be about equal to calling a function from a library. Too small to be worth considering unless it was done in a loop.

That's preposterous. Turning his game into what amounts to a library to suit you? Not likely.

IPC is too slow (memory access synch is far too slow) and using it as a library would require a massive restructuring that's not in any way, shape, or form to Toady's benefit.

And Andir, you're overgeneralizing your speciality into a field that doesn't make any sense. Request-based client/server models are used in the web and on disparate hosts because they have to be. Doing it locally would be stupid (in addition to requiring a massive restructure that wouldn't benefit Toady).

Multithreading will be helpful, but synchronization primitives are costly and I strongly doubt that reworking the entire logic end of the game to make a library, just in order to appease a few people, makes any sense.

[Andir]

And Andir, you're overgeneralizing your speciality into a field that doesn't make any sense. Request-based client/server models are used in the web and on disparate hosts because they have to be. Doing it locally would be stupid (in addition to requiring a massive restructure that wouldn't benefit Toady).

Multithreading will be helpful, but synchronization primitives are costly and I strongly doubt that reworking the entire logic end of the game to make a library, just in order to appease a few people, makes any sense.

When you deal with threads, other than shared memory models, the easiest way to communicate is via client/server like methods.  Unless you're gunning for race condition bugs?  I was clarifying how it could be done.  Not how it should be done.  (Although, like the interface, if it's not decided up front it's not likely to ever get done.)

[Sindai]

The only performance-intensive real time games I can think of that allow extensive interface modding do it via scripting language. Lua is the first one that springs to mind, since both Supreme Commander and World of Warcraft use it.

There might be a counterexample out there, but the body of existing games that allow this kind of thing seem to support Blacken on this. Has anybody actually used the multiprocess or API models for this before?

[Dr. Melon]

I think a "skin" system would work.

Then people could design their own UI's and such, but Toady still decides the base framework and how far that can go.

[isitanos]

Or do you expect Toady to resend all the creatures and items every frame? Or wait no, the client side caches all that unchanged information meaning that it doubles the memory usage.

It's not necessary to resend everything every frame: the client asks each frame for the specific region of the game he wants to render, and gets back only the necessary info, i.e. the visible objects and creatures in that region. This keeps Toady's data structures nicely out-of-view, and the client doesn't even know about hidden chasm creatures and HFS until they are discovered. This means clients that offers a larger 2d view or a 3d view would run slower, but that sounds ok to me.

I'm thinking of very nice features that third-party clients could bring...
- If someone re-makes a pure curses client, that means we could play by telnet/ssh: that would be great for succession games, we could watch other players run the fortress, and everybody would connect to the same computer to play.
- Auto-pause and pausing only on selected events could be handled by the client. Actually, the server should probably never pause the game unless requested by the client.
- (From the other interface thread) We wouldn't have to pick between an interface that pauses when you right-click and one that doesn't: with a variety of third-party clients, we'll have the occasion of trying out different interfaces to find one that suits us the best.
- ...

Right now I'm trying to find examples of games that have third party clients, and the effect they had on the game's development. Does anybody have a good case to study?

[Frobozz]

That's preposterous. Turning his game into what amounts to a library to suit you? Not likely.

Not exactly. More the reverse where the game uses an interface library to handle the input/output much in the same way it currently uses a library/libraries to do it now. It would simply become a matter of replacing the interface library with the one you wanted at the time of compile (or runtime by replacing the shared library with the one you want).

IPC is too slow (memory access synch is far too slow) and using it as a library would require a massive restructuring that's not in any way, shape, or form to Toady's benefit.

A massive restructuring? You say this as though you know the internals of Dwarf Fortress. Beyond Toady mentioning details like various things being hard-coded, you're just guessing. An educated guess one could say, but a guess nonetheless.

Also I fail to see how it would not benefit Toady. This method would effectively allow him to separate the game logic from the interface. This would allow for removal of any limitations the logic expects from the interface and thus any limitations the interface expects from the logic. It would allow him to change the interface when and how he wants without spending months reworking the logic and trying to fix all the bugs that come from reworking it.

And why do you think speed is such an issue? If syncing things is such an issue, you could go the route of having the game thread only process data when told to by calling an update function.

[Jifodus]

- If someone re-makes a pure curses client, that means we could play by telnet/ssh: that would be great for succession games, we could watch other players run the fortress, and everybody would connect to the same computer to play.

Actually with some careful hacks, it is possible to have a pure telnet/curses client right now. I "know" where the screen buffer is in memory (I just haven't really experimented with it so I'm not 100% sure of it's format). Plus faking windows input messages will allow the game to receive input. I just haven't been motivated to start working on a telnet game server for DF.

For those interested here's the address & offset and code block information:

Spoiler (click to show/hide)

Code: [Select]

The address of the screen object
.text:00443158 008 B8 88 37 6A 01                                mov     eax, offset screen ; 016A3788
.text:0044315D 008 88 1D E0 3F 6A 01                             mov     screen_bold, bl
.text:00443163 008 E8 38 C0 20 00                                call    put_c_string_screen

The code that places a character on the screen
.text:0064EFA0                                   ; ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦ S U B R O U T I N E ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦
.text:0064EFA0
.text:0064EFA0
.text:0064EFA0                                   put_char_screen proc near               ; CODE XREF: sub_443120+133p


(Ok, post preview doesn't want to work, I'll just post it, ignore any flaws please.)

[dreiche2]

Right now I'm trying to find examples of games that have third party clients, and the effect they had on the game's development. Does anybody have a good case to study?

This might be a somewhat different example in several respects, but civilization is very modable: see here. There are three levels, XML for the data (much like Toady's text files now), Python for a lot of what happens during the game including aspects of the interface, and even a SDK to modify the "DLL source code" of the game. Of course, making the game that way probably wasn't trivial. For the Python part, they used the so-called Boost libraries, which are supposed to simplify producing a Python interface to your c++ program. Myself I don't have any experience with them.

I did a little bit of Python scripting for Civ4 myself (map generation), so I can describe how it looks in practice: Basically, you had certain functions such as "generate_terrain" or "place_civilizations". The way the engine used these functions was not accessible via Python (e.g. when they are called etc), but you could implement what they actually do in Python (e.g. some people simulated plate tectonics to generate the terrain). Moreover, you had predefined functions you could call to request data from the engine (e.g. number of players etc.).

As far as I understand, Boost basically helps you creating wrappers around your c++ functions to be accessible from Python or vice versa.

Maybe this would be a starting point *if* Toady ever wants to explore the subject (and yes, there might be more important things). Use boost so that one has in Python two functions, "draw" and "getInput", which are called from the engine. "Draw" can access only the visual info which would normally be displayed, and "getInput" channels what would be key commands. With that you couldn't do much more than say maybe implement mouse support or rearrange some basic stuff, but it might be something to try out.

[Andir]

Boost does WAY more than that:
http://www.boost.org/doc/libs/1_35_0/libs/libraries.htm

Everything from I/O to threading to cross language stuff you mentioned.

[Symmetry]

You'd only free up maybe 10ms of CPU time (read the Kobold Quest source code, it has DF's rendering method), which on most computers is at most 1 more frame. Also, you'd probably lose 20-50ms just by sychronizing the data accesses.

I think the fact that reducing DFs graphics framerate increases the game framerate so much shows theres lots of room for optimisation in this area.  If the rendering was efficient reducing your graphics framerate would have no noticeable effect on the game update rate.  From looking at the kobold quest source it seems like it's making a lot of calls to the driver, this is usually slow.
As for threading, if toady doesn't know how to do it I'm glad he has the sense to stay away from it.  It isn't easy at all.

Boost is great, some of the libs are less great.  The graph one makes my head hurt.

[Tormy]

Toady might just ignore the whole discussion

He should.

[isitanos]

- If someone re-makes a pure curses client, that means we could play by telnet/ssh: that would be great for succession games, we could watch other players run the fortress, and everybody would connect to the same computer to play.

Actually with some careful hacks, it is possible to have a pure telnet/curses client right now. I "know" where the screen buffer is in memory (I just haven't really experimented with it so I'm not 100% sure of it's format). Plus faking windows input messages will allow the game to receive input. I just haven't been motivated to start working on a telnet game server for DF.

Amazing. Done right, that could become huge. Dungeon Crawl Stone Soup's telnet server is quite popular, so I can't imagine what would happen if we could start DF telnet servers.
I'm really not into memory hacking and reverse engineering, but you make me wish I was.

[Anu Necunoscut]

Hacking the memory to suborn DF's inner workings to your whims, against Toady's express wishes?  To me that's monstrous ingratitude--"thanks for the years you poured in, but based on my few months of play -I'm- the one who knows where this game should go and I'll take it there regardless of your plans."  Seriously?  3dwarf represents the sort of memory hacking by modders I don't mind so much--discreet, useful, black-box, and entirely peripheral.  This other form?  Well, when you're invited as a guest to someone's house, you don't surreptitiously rifle through the host's private possessions because you think you can arrange them better.

[ervill]

I dont think memory access is needed for this particular field of application because it should be no problem to parse the opngl output image for the tileset. Even so, the whole idea would be a too complicated undertaking, i think.