Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Idiom]

Will someone please finally start looking into or start an experiment with streaming DF? There's ANOTHER "DF on X-portabledevice" thread. Lets start discussing some solutions.

Obviously we really can't crunch path finding and such on a ULP device. As we've told every person already. Streaming it from a more powerful computer, unless someone has a better idea, sounds like a possibility to me. All you need is a two part program.

One half breaks down the output. Baughn is screwing with the SDL of the rendering isn't he? Hell, we have the rendering source available. All we need to do is get it to output a small packet containing all the ASCII symbols that it is rendering to be reassembled in a grid by a ULP device. For special tilesets, recognize the particular tile and output the tile IDs in the same manner to be sent to a ULP loaded with the same tileset. It doesn't need to stream the full 100 FPS much as the game currently does on capable systems. GFPS cap anyone? I run mine at eight and I can't even notice the difference from a true 100. Stream it over the net to be reassembled into a grid on your netbook at the cafe once every eight game frames or so.

The other half would be a 3rd party interface for devices with only touchscreens to show buttons alongside the grid which when pressed tell the host computer to macro the particular key. IE the "Designate" button translates as a "d" keypress on the host. Most ULP devices can't display anywhere near the minimum grid size, and this is where the real problem is, even if you use 8x8 tiles (would need a 640x200 screen?) and then room for buttons if it's a touch screen. A netbook could do that, and it would have it's own keyboard so no need for the touchbuttons. Toady would have to change the minimum display grid size to something much smaller than it currently is (at least 32x24 for the DS running an 8x8 tileset with grid on the topscreen and buttons on the touchscreen. At least 40x40 for the Iphone leaving a 160x320 strip for touch buttons which may be too small). Obviously we can't cram all the buttons onto the screen at once, but I think people would be fine with scrolling. Some phones with full keyboards could do without the touchbuttons and use the whole screen for the grid.

So now we're looking at the most trouble we're asking is a small tweak from Toady One. I don't think it's too much, and if there's a price (5-10$?) put on the Iphone App with the proceeds sent to Adams himself I think he could find that very worthwhile.

Now just how do we send the data to the ULP device? The netbooks/crappy laptops could just receive over the internet via wifi or wireless air card in a traditional manner. A DS would need some good homebrewers to use the DS's built in wifi. Something like a phone though could potentially receive with much better coverage than just a wifi device. Take advantage of unlimited texting plans and text ACSII at 8 messages a second? How would it send output via texting to a computer? Take advantage of unlimited minutes and send digital signals at 8 signal pulses a second while sending digitized output signals? Will we have to do with less than 8 frames sent a second? I really don't know about the phones doing DF, but I'm fairly certain we can at least get wifi devices like netbooks/crappy laptops doing this, and that wouldn't even require a minimum grid change on Toady's part.

[N3X15]

I have no idea what the backend would look like.  However, one may be able to do a telnet-like session between the server and the client, where each client merely tells the server which actions it wishes to perform, and the server sends map updates.

[Idiom]

Based on your name, I presume you know a hell of a lot more than I do about programming or networking. Continue to ramble on the subject, please, because I've said about all I can and I know no more than that.

[Jay]

There's a protocol that you should be looking into..  Virtual Network Computing (VNC for short)
It's already present on the aforementioned Nintendo DS as DS2Win/Win2DS and the like.
For Windows and Linux, you can try www.tightvnc.com

[peterix]

VNC is slow.

Have a server app that periodically reads the DF screen directly from DF memory and send it to the client(s). It should also receive input from the client. Compression is a must. Make the server also start DF when it's not running - and make it cross-platform. It should run as a daemon/system service.

Client has input, networking and display - it should be possible to reuse the libgraphics library of DF. Also cross-platform.

As a bonus, multiple clients could be connected to one fort. One interactive, others as spectators... maybe allow them to take turns as the active player? Give them some limited ability to designate stuff by using memory hacking? It could be huge.

[Idiom]

As a bonus, multiple clients could be connected to one fort. One interactive, others as spectators... maybe allow them to take turns as the active player? Give them some limited ability to designate stuff by using memory hacking? It could be huge.

YES

You up for it? That's why this thread is here.

[Thndr]

And if you're going to directly read the DF screen from memory and send it over a network, why not have the client have a local tileset to reduce the load over the network, therefor a broadcast wouldn't take up much bandwidth.
I'm pretty sure most devices can at least draw tiles and take input.

Also, if the server sends the memory over the network, the client could also be one of the 3D rendering clients that could make it that people can watch DF streaming in 3D. (And if the client also can relay input commands to the server, make DF playable 3D without worrying about how slow DF will run while running the 3d client).


Streaming someone make a fortress using a 250 FPS cap on a super mega ultra spectacular uber machine that can handle a 6x6 map(or larger!), all features, 250 dwarfs.

That would be epic.


But you also get a question about users with different resolutions and what to draw and such, would the client be forced at a specific resolution or something?

And for multiple users getting control, I think that if there becomes a 3rd party program that does this, the server should have an admin that chooses people, or have the current chosen player be able to choose the next player.

---

Personally I find this a decent idea, although would be a lot better if implemented first party sometime in the future maybe. (Although then it'd probably turn into a DF MMO in which people do not like the idea of, even if it's just multiple fortresses being ran at once) But any discussion about DF that way probably would be better in the suggestions forum

[Arkose]

As a bonus, multiple clients could be connected to one fort. One interactive, others as spectators...

Something like this, even if it were just for regular computers and not handheld-focused, would be amazing. While being able to hand over control and such would be beyond quality (tm), just having a way to stream the DF display to spectators would be a real step up over watching someone stream video of a DF game on a videochat service.

(There should, of course, be some kind of simple DF chatroom bundled with the DF streamer.)

How well documented is the reverse-engineering of DF's memory model? (I know there are several 3D visualizers.) At a bare minimum, you would want to be able to scrape the necessary details of what the host-player is viewing. (A more ambitious project might let connected spectators look around the map by examining the map memory of the places they want to independently view.)

I think something like that should probably be the first step. It would be useful in its own right, and also lay the foundations needed for games viewed on one computer but hosted on another (either for use with portables calling home to a home server, or as a spiffy new way to handle succession games).

[darius]

got crackin'

results: 0x00664440 start of buffer.
tile size: 4 bytes

• tilenum
• color1
• color2
• something

format is first height then width. Buffer is allocated for maximum size so each column is 1024 bytes away from other.

PS will try to make something similar to remote viewer...

[Arkose]

results: 0x00664440 start of buffer.
tile size: 4 bytes

• tilenum
• color1
• color2
• something

What tools did you use to determine this? I'd like to see if it is the same on OSX, since that's what I happen to have available to me at the moment. (The only thing I would expect to be different would be the buffer offset.)

-EDIT- Removed incorrect speculation

[Slogo]

tilenum is going to be the ascii used I believe. What kind of values does 'something' have. That's going to tell us a lot more about what's there.

[Armok]

Buffer is allocated for maximum size so each column is 1024 bytes away from other.

My first thought upon reading this was "Toady is such an awesome programmer, he couldn't possibly waste memory like-" then I realized it was ASCII tiles, not pixels, and thus probably is negligible.
I'm tired and posting random stuff I really shouldn't.

[peterix]

And if you're going to directly read the DF screen from memory and send it over a network, why not have the client have a local tileset to reduce the load over the network, therefor a broadcast wouldn't take up much bandwidth.
I'm pretty sure most devices can at least draw tiles and take input.

I'm thinking about capturing the tiles and their colours before they are rendered. There would probably be problems with using the 'graphics' mode instead of plain ascii because of all the custom graphics for creatures and stuff like that. Second route is to use the same algorithm used for DFMA maps. Only make it capture images off the screen/df window and stream them instead of producing files.

Also, if the server sends the memory over the network, the client could also be one of the 3D rendering clients that could make it that people can watch DF streaming in 3D. (And if the client also can relay input commands to the server, make DF playable 3D without worrying about how slow DF will run while running the 3d client).

This is happening only when the server does memory hacking. Also, the stream has to be aggressively compressed (only send real changes, omit what the clients don't see, etc.). I'm working on a DF memory hacking library and it's progressing quite well. Still needs a lot of work to allow even basic uncompressed streaming tho... It's currently used as the memory extraction component of khazad.

But you also get a question about users with different resolutions and what to draw and such, would the client be forced at a specific resolution or something?

There's a clear distinction between the interactive player (the one getting the 'real' game) and the spectators/secondary players.
The main player has to use the current df interface with all its inherent limitations. So, no 3D - or maybe as a secondary display. You can't control the game all that well using memory hacks.
Spectators/secondaries can get the copy of the main stream or alternatively some custom memory-hacked view. Their ability to control the game is limited and probably won't see some of the details while using memory hacks.

And for multiple users getting control, I think that if there becomes a 3rd party program that does this, the server should have an admin that chooses people, or have the current chosen player be able to choose the next player.

Yes, there should be some considerations about security and stuff like that. It's mainly a problem of designing the networking part well.

Personally I find this a decent idea, although would be a lot better if implemented first party sometime in the future maybe. (Although then it'd probably turn into a DF MMO in which people do not like the idea of, even if it's just multiple fortresses being ran at once) But any discussion about DF that way probably would be better in the suggestions forum

Can't see that MMO thing happening any time soon. You would need clustering/lots of metal to run the whole world at the same time. Also, DF would have to change significantly.

It's a lot of work and I could use some more dedicated people working on stuff. I'm just a single guy - and learning as I go

How well documented is the reverse-engineering of DF's memory model? (I know there are several 3D visualizers.) At a bare minimum, you would want to be able to scrape the necessary details of what the host-player is viewing.

From the raw map data, I have the map blocks with most of their properties, most of the matgloss data, shrubs, trees, and very basic buildings/zones/stockpiles extraction. I'll be expanding on this further and adding creatures, language, items and objects-holding-items/other objects. This will take a lot of time.

got crackin'

results: 0x00664440 start of buffer.
tile size: 4 bytes

• tilenum
• color1
• color2
• something

format is first height then width. Buffer is allocated for maximum size so each column is 1024 bytes away from other.

PS will try to make something similar to remote viewer...

Awesome. For which version of DF is this? I'll be more than happy to find offsets for all the current linux/windows versions when I know where and what to look for

PS: I'm really busy with khazad and its memory extractor. I'll probably add screen extraction to the library but it will take some time to finally separate it from khazad.
PPS: It's all in plain old C++.

[Arkose]

Buffer is allocated for maximum size so each column is 1024 bytes away from other.

My first thought upon reading this was "Toady is such an awesome programmer, he couldn't possibly waste memory like-" then I realized it was ASCII tiles, not pixels, and thus probably is negligible.

256 x 256 x 4 bytes = 262,144 bytes
80 x 25 x 4 bytes = 8,000 bytes

So in all, it comes down to ~0.25 megabytes of memory wasted for people playing with the standard display size. Not THAT bad, although it's an obvious optimization that Toady might want to look into sometime. (In addition to freeing up memory, you would probably end up with fewer memory cache misses, which can be a big win.)

I'd like to know if the whole 256 x 256 buffer gets updated with display information even when the viewport dimensions are smaller. That would be a big waste of processor time. (My guess is that it doesn't.)

[peterix]

I'd like to know if the whole 256 x 256 buffer gets updated with display information even when the viewport dimensions are smaller. That would be a big waste of processor time. (My guess is that it doesn't.)

I bet it's only completely rewritten when you move the view around (and even then, only the view part gets updated). Otherwise it's just collecting the changes.