Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[MrWiggles]

Yes and No. The game has a good average of performance. Some release are slower then others. The last big release had a slower World Gen, but it's iterated release have relived that somewhat, though it's exposed the 32bit limitation of DF.

When we got the more extensive caverns, performance stayed more or less the same, even though the area of Play increased several folds.

And yea, Pathing is the more popular reason for FPS death, but Path Finding is a pretty constant stress and regular. What always really killed late game forts is the number of Items (and Items are pretty much everything. Clothing, Rocks, Splatters), and revealed Tiles. The number of Items increases several folds as each in game year your game progresses.

This is probably why Nano Forts are so healthy for FPS, it dramatically decreases the Items that can be in the game.

[runlvlzero]

Also, I'de like to clearly restate what I think some people are stating.

Memory leaks will make it appear like the game is badly handling each tile.

Memory leak could be coming from anywhere, almost all big programs I have ever used have had some memory leak B.S. at some point.

It could be the windows libraries SDL is calling that is causing over time memory leakage. It could be shit in directx (if the SDL graphics and sound are being accelerated through that). It could be a billion billion things...

To be honest though I don't think DF has bad memory leaks (or any at all, not any that I would acknowledge) because I can run it for 18+ hours without it randomly crashing and throwing exceptions.

Ive never had DF crash from just sitting and running. Which speaks great volumes that its not leaky memory.

Programs with "known" (acknowledged by the dev) memory leaks I've used, all usually crash in under a few hours on my system.

I think DF rocks in this department. So back to my original theory that there's a real reason behind it and its not a cause for performance issues.

[expwnent]

Starting memory usage: ~380,000K and wobbly
After reveal: the same
After digging out a z-level: 397,000K and wobbly

~44 KB/tile

This is per tile dug out, not per tile in existence.

A memory leak is not always random. It could be that there's a memory leak that happens only when you dig out a tile. Or it might not be a memory leak at all. It could just be a bad memory use design. Thinking about it my best wild guess is that each tile contains a vector<> that is initially very large, or contains very large things, when it should just be initially small. I'll speculate more later on what could be stored per tile. I have to do work now.

Edit: I was going to try checking Undergrotto's memory usage, but according to dffd, it uses 0.28.181.40d, which is a fairly old version. It's possible that it had less memory usage. I've actually gotten DF to crash before just by digging out tiles. Though I have not tried it, I strongly suspect that Undergrotto would be impossible to recreate with today's version (0.34.07).

Edit 2: Screw work. It looks like saving, exiting, and reloading dramatically reduces memory usage. This might not be so bad after all, but it's still crazy high memory usage.

[runlvlzero]

Starting memory usage: ~380,000K and wobbly
After reveal: the same
After digging out a z-level: 397,000K and wobbly

~44 KB/tile

This is per tile dug out, not per tile in existence.

A memory leak is not always random. It could be that there's a memory leak that happens only when you dig out a tile. Or it might not be a memory leak at all. It could just be a bad memory use design. Thinking about it my best wild guess is that each tile contains a vector<> that is initially very large, or contains very large things, when it should just be initially small. I'll speculate more later on what could be stored per tile. I have to do work now.

Edit: I was going to try checking Undergrotto's memory usage, but according to dffd, it uses 0.28.181.40d, which is a fairly old version. It's possible that it had less memory usage. I've actually gotten DF to crash before just by digging out tiles. Though I have not tried it, I strongly suspect that Undergrotto would be impossible to recreate with today's version (0.34.07).

Edit 2: Screw work. It looks like saving, exiting, and reloading dramatically reduces memory usage. This might not be so bad after all, but it's still crazy high memory usage.

Big vectors could be the case, but it takes the same time to access data pointed too in memory regardless how much space is reserved to point too. (to a point)... but if its 5k stored, and you point somewhere in that 5k, it takes the same time as if 500k is stored and your still looking for 8 bytes in that 500k, as long as its already addressed. Allot of that data might be addresses, meaning efficiency for whatever... In other words datastructure is the key not size... its how databases with trillions to the trillions of entries work.

[MarcAFK]

Operation FPS bomb has shown that memory and FPS issues seemingly get 99% fixed when you save and reload.
It's like everything you do ingame causes increasing memory usage and lag which gets cleared when refreshed.
We haven't found the cause of fort ending lag yet though.

[Lord Dullard]

A memory leak is not always random. It could be that there's a memory leak that happens only when you dig out a tile. Or it might not be a memory leak at all. It could just be a bad memory use design.

Probably this.

I think it DOES have something to do with pathfinding, but not in the pathfinding algorithm itself. Rather, it's in the way that DF stores pathfinding information about tiles, since Toady has stated previously that DF has some kind of weird way of storing tile-related info. I suspect the continual sorting of those tiles relevant to the various creatures/entities on the map causes increasing amounts of lag.

[GhostDwemer]

A memory leak is not always random. It could be that there's a memory leak that happens only when you dig out a tile. Or it might not be a memory leak at all. It could just be a bad memory use design.

Probably this.

I think it DOES have something to do with pathfinding, but not in the pathfinding algorithm itself. Rather, it's in the way that DF stores pathfinding information about tiles, since Toady has stated previously that DF has some kind of weird way of storing tile-related info. I suspect the continual sorting of those tiles relevant to the various creatures/entities on the map causes increasing amounts of lag.

Thirding the "probably this" with the caveat that "bad memory design" might actually mean "optimized to use more memory and less processor." There are plenty of techniques that trade off memory usage for speed,  like keeping paths or path segments cached, unrolling loops, and so forth. I think anyone speculating about performance should be required to disclose their skill level in computer science. If you would just start your speculations with something like "I've never programmed a computer before, but..." or "My name's George Broussard..." it would really help. For example, I've been using computers since I was six, in 1976, programming since I was nine, and working in systems administration since I was nineteen. With my level of expertise, and no access to the source code, this is as much as I'm willing to speculate.

[runlvlzero]

A memory leak is not always random. It could be that there's a memory leak that happens only when you dig out a tile. Or it might not be a memory leak at all. It could just be a bad memory use design.

Probably this.

I think it DOES have something to do with pathfinding, but not in the pathfinding algorithm itself. Rather, it's in the way that DF stores pathfinding information about tiles, since Toady has stated previously that DF has some kind of weird way of storing tile-related info. I suspect the continual sorting of those tiles relevant to the various creatures/entities on the map causes increasing amounts of lag.

Thirding the "probably this" with the caveat that "bad memory design" might actually mean "optimized to use more memory and less processor." There are plenty of techniques that trade off memory usage for speed,  like keeping paths or path segments cached, unrolling loops, and so forth. I think anyone speculating about performance should be required to disclose their skill level in computer science. If you would just start your speculations with something like "I've never programmed a computer before, but..." or "My name's George Broussard..." it would really help. For example, I've been using computers since I was six, in 1976, programming since I was nine, and working in systems administration since I was nineteen. With my level of expertise, and no access to the source code, this is as much as I'm willing to speculate.

Amen, an my name is George Broussard!

Just kidding though, I'm a derp with 7 years experience in IT doing various things. Programmaticly similar experience was getting windows to load bitmaps of images to scan the pixels with algorithms (I didn't do the mathy algorithms). I'm self taught, I can guarantee all my terminology is ate up, but I do know a bit about memory management.

So yeah, Ive played with that data structure, and read at least enough to understand whats going on.

Bitmap files are huge tables of information, but they are not super compressed jpg's or PNG's, yet that data structure is still used to do image manipulation and to actually open up and display most images, even jpg's. JPG is just a storage format, while bitmap has been a working format, but I'm sure there's more advanced stuff now. They are also used for other things as well, not just images for those nerdy people out there who might jump down my throat for not mentioning that...

[SAFry]

Think the large address extender would help? I mean I have 4 gig of RAM but 1926 is being used by the O/S and a couple browser windows. So if DF can only see 2 gig of memory addresses does that mean it can only 'see' the last 74 meg of RAM? Probably not, but it would have to leak a hell of a lot to get through 2 gig of RAM.

Having said that I just patched the .exe and launched DF and was getting pretty much the same FPS as I was getting at the end of play last night so that would suggest that it hasn't done anything to help RAM issues or RAM isn't the issue.

[runlvlzero]

Think the large address extender would help? I mean I have 4 gig of RAM but 1926 is being used by the O/S and a couple browser windows. So if DF can only see 2 gig of memory addresses does that mean it can only 'see' the last 74 meg of RAM? Probably not, but it would have to leak a hell of a lot to get through 2 gig of RAM.

Having said that I just patched the .exe and launched DF and was getting pretty much the same FPS as I was getting at the end of play last night so that would suggest that it hasn't done anything to help RAM issues or RAM isn't the issue.

The LAA patch is only useful for really long world gens. After a few hours running I haven't gone past 2gigs usage myself, but people who let it gen for a day or two probably do see bigger memory footprints then

[MarcAFK]

Very large embarks also require the patch, i was hitting the limit frequently with 8x8 embarks

[runlvlzero]

OMG dude do you have a quantum singularity for a CPU?

[MagmaMcFry]

Starting memory usage: ~380,000K and wobbly
After reveal: the same
After digging out a z-level: 397,000K and wobbly

How about you run a control world, spending the same amount of time without digging out a Z-level? Try making two or three small stockpiles that feed from each other to keep your dwarves busy, and put one dwarf into a custom workshop appearifying a Z-level's worth of stone in the meantime.

[Talvieno]

A memory leak is not always random. It could be that there's a memory leak that happens only when you dig out a tile. Or it might not be a memory leak at all. It could just be a bad memory use design.

Probably this.

I think it DOES have something to do with pathfinding, but not in the pathfinding algorithm itself. Rather, it's in the way that DF stores pathfinding information about tiles, since Toady has stated previously that DF has some kind of weird way of storing tile-related info. I suspect the continual sorting of those tiles relevant to the various creatures/entities on the map causes increasing amounts of lag.

I think this may make more sense as well.

[Lord Dullard]

A memory leak is not always random. It could be that there's a memory leak that happens only when you dig out a tile. Or it might not be a memory leak at all. It could just be a bad memory use design.

Probably this.

I think it DOES have something to do with pathfinding, but not in the pathfinding algorithm itself. Rather, it's in the way that DF stores pathfinding information about tiles, since Toady has stated previously that DF has some kind of weird way of storing tile-related info. I suspect the continual sorting of those tiles relevant to the various creatures/entities on the map causes increasing amounts of lag.

Thirding the "probably this" with the caveat that "bad memory design" might actually mean "optimized to use more memory and less processor." There are plenty of techniques that trade off memory usage for speed,  like keeping paths or path segments cached, unrolling loops, and so forth. I think anyone speculating about performance should be required to disclose their skill level in computer science. If you would just start your speculations with something like "I've never programmed a computer before, but..." or "My name's George Broussard..." it would really help. For example, I've been using computers since I was six, in 1976, programming since I was nine, and working in systems administration since I was nineteen. With my level of expertise, and no access to the source code, this is as much as I'm willing to speculate.

Hm. As for myself, I am no expert, although I've taken a couple of years of college courses towards a Comp Sci degree (none of the really advanced stuff yet). I do know how to code in a few languages and I'm currently working on the game linked in my signature, and my experience in doing that is mostly what I'm basing my conjecture off, since it's tile-based.