Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: I think I've found something that creats lag. (38c)  (Read 1624 times)

Chrispy

  • Bay Watcher
    • View Profile
I think I've found something that creats lag. (38c)
« on: April 21, 2008, 04:30:00 pm »

I was just defraging my computer, and I thought to look at the smallest files with more than 100 fragments.  My gamelog.txt file for dwarf fortress was near the top, with 752 fragments in a 3109kb file.  Considering how it's used in-game, and that it takes me nearly a minute to open it in wordpad.exe, perhaps it would be an idea to somehow make it so it didn't massively fragment itself?  

Duplicating the file, deleting the original and renaming the copy to the original has improved my fps a small amount, although that could just be me being biased.

Logged
Quote from: Toady
"This suggestion is very broad."
≡«+large yellow diamond+»≡
On the item is an exceptionally designed image of a diamond in yellow diamond.

zagibu

  • Bay Watcher
    • View Profile
Re: I think I've found something that creats lag. (38c)
« Reply #1 on: April 21, 2008, 05:13:00 pm »

I think the framerate is not ifluenced by disk access at all, since all data manipulations are done in memory, once the map/save has finished loading. It would be really really bad for framerate, if it were, since disk access times are in the range of 10ms...

[ April 21, 2008: Message edited by: zagibu ]

Logged
99 barrels of beer in the pile
99 barrels of beer!
If some dwarves know the way to the pile
0 barrels of beer in the pile!

Derakon

  • Bay Watcher
    • View Profile
Re: I think I've found something that creats lag. (38c)
« Reply #2 on: April 21, 2008, 05:24:00 pm »

quote:
Originally posted by zagibu:
<STRONG>I think the framerate is not ifluenced by disk access at all, since all data manipulations are done in memory, once the map/save has finished loading. It would be really really bad for framerate, if it were, since disk access times are in the range of 10ms...

[ April 21, 2008: Message edited by: zagibu ]</STRONG>


This kind of issue should be handled entirely by the OS, and a decent write-through cache should handle this just fine. You'd only run into problems if you were out of RAM.
Logged
Jetblade - an open-source Metroid/Castlevania game with procedurally-generated levels

ikkonoishi

  • Bay Watcher
    • View Profile
Re: I think I've found something that creats lag. (38c)
« Reply #3 on: April 21, 2008, 05:39:00 pm »

quote:
Originally posted by zagibu:
<STRONG>I think the framerate is not ifluenced by disk access at all, since all data manipulations are done in memory, once the map/save has finished loading. It would be really really bad for framerate, if it were, since disk access times are in the range of 10ms...

[ April 21, 2008: Message edited by: zagibu ]</STRONG>


Event log is updated in nearly real time. If you have two screens you can see the last modified time update as DF runs.

Logged
Our Dwarven instincts compel us to run blindly towards disaster in case there may be a ☼<☼giant cave spider silk sock☼>☼ lying around.

Jay

  • Bay Watcher
  • ☼Not Dead Yet☼
    • View Profile
Re: I think I've found something that creats lag. (38c)
« Reply #4 on: April 21, 2008, 06:45:00 pm »

quote:
Originally posted by ikkonoishi:
<STRONG>

Event log is updated in nearly real time. If you have two screens you can see the last modified time update as DF runs.</STRONG>


You don't need two screens.
Windowed mode.
Logged
Mishimanriz: Histories of Pegasi and Dictionaries

ikkonoishi

  • Bay Watcher
    • View Profile
Re: I think I've found something that creats lag. (38c)
« Reply #5 on: April 21, 2008, 08:23:00 pm »

quote:
Originally posted by jaybud4:
<STRONG>You don't need two screens.
Windowed mode.</STRONG>

Thought about that as I posted didn't think it worth editting to add.

Logged
Our Dwarven instincts compel us to run blindly towards disaster in case there may be a ☼<☼giant cave spider silk sock☼>☼ lying around.

Tacroy

  • Bay Watcher
    • View Profile
Re: I think I've found something that creats lag. (38c)
« Reply #6 on: April 22, 2008, 03:33:00 am »

Just because the file you open updates doesn't mean that it's actually been written to disk; you could be opening the version of it in memory. Of course, it's possible that Windows is retarded enough for that to really be the case, but I wouldn't know. In any case, keeping such things defragmented is honestly not DF's responsibility; the OS should be handling that.
Logged

falsedan

  • Bay Watcher
    • View Profile
Re: I think I've found something that creats lag. (38c)
« Reply #7 on: April 24, 2008, 11:49:00 am »

yeah windows will cache the write until an opportunity arises to sync it to disk, also the metadata so explorer windows will show the last updated time using the information from the cache

dwarfort.exe never reads from the log so it doesn't matter how fragmented it is, at each sync the OS will find the last block in the file that still has space and put the data there (or allocate a new block somewhere for it if the last block is full)

writing to fragmented files is fine; reading them is sloooooow

Logged

Chrispy

  • Bay Watcher
    • View Profile
Re: I think I've found something that creats lag. (38c)
« Reply #8 on: April 25, 2008, 12:27:00 am »

I find it very peculiar that the file had something like 1 fragment for ever 4.1Kb.  Considering that the smallest available block on my disk is 4kb, that seems awfully strange.  It would mean, as far as I can see it, that DF is trying to append to the file as often as possible.  I guess that makes sense, as it's an crash-log file.  I suppose I'll just continue manually coalescing it on my own then.

I also noticed that when I started a fortress, dug around a bit, caused a small cave-in, and then quit, that map file ended up broken into 53 fragments after 1 min of play.  I've got a large number of decent sized gaps on my hard drive, and I have difficulty seeing why these files are getting as fragmented as they are.

Alas, I am not a skilled programmer, I just know enough to get into trouble.

Logged
Quote from: Toady
"This suggestion is very broad."
≡«+large yellow diamond+»≡
On the item is an exceptionally designed image of a diamond in yellow diamond.

axus

  • Bay Watcher
  • Axe Murderer
    • View Profile
Re: I think I've found something that creats lag. (38c)
« Reply #9 on: June 16, 2008, 11:33:34 am »

Surely the game is writing to disk, but I don't notice the hard drive being pegged like its writing all the time.  Are there options to turn off the logging?  It would be interesting to see if that helped.
Logged

Shades

  • Bay Watcher
    • View Profile
Re: I think I've found something that creats lag. (38c)
« Reply #10 on: June 20, 2008, 11:37:16 am »

Just ask Toady if he's doing a file flush() operation after each write :)

That said I've worked on many systems that do similar things for logging purposes, windows (despite it's flaws) is capable of handling the disk caching without problems (in fact memory mapped files are one of the fastest ways to transfer information around... not that anyone really cares :P)
Logged
Its like playing god with sentient legos. - They Got Leader
[Dwarf Fortress] plays like a dizzyingly complex hybrid of Dungeon Keeper and The Sims, if all your little people were manic-depressive alcoholics. - tv tropes
You don't use science to show that you're right, you use science to become right. - xkcd