Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2

Author Topic: A sort of a program memory backupy thing...  (Read 1963 times)

Schilcote

  • Bay Watcher
    • View Profile
A sort of a program memory backupy thing...
« on: April 10, 2010, 08:52:01 pm »

I like to play a game that was ported to windows from PocketPC. It crashes almost constantly. I was wondering if there was any sort of program that could write a backup of the program's memory to hard disk every so often, so that when it crashes I can simply reload the data into memory and continue.
Logged
WHY DID YOU HAVE ME KICK THEM WTF I DID NOT WANT TO BE SHOT AT.
I dunno, you guys have survived Thomas the tank engine, golems, zombies, nuclear explosions, laser whales, and being on the same team as ragnarock.  I don't think something as tame as a world ending rain of lava will even slow you guys down.

psyn

  • Bay Watcher
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #1 on: April 10, 2010, 09:35:18 pm »

Impossible. Processes cannot be started from mapped memory. Heaps are deallocated on program exit. It can't reallocate them in the same memory space. File/device/OS handles would become invalid. Internal memory offsets (between threads and DLLs) become invalid. Any other state the OS records, like critical sections, mutexes, windows, messasing, threads/semaphores, timers, and two dozen other things, are invalid. And I'm forgetting or unaware of so much more.
Logged

Schilcote

  • Bay Watcher
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #2 on: April 10, 2010, 10:06:10 pm »

So what you're saying is that you can't just push saved data into memory and expect it to run because programs depend on other bits of data stored in memory that will have become invalid?

Why can't I just save the data portion? The program doesn't need to be saved, just the numbers that it writes down...
Logged
WHY DID YOU HAVE ME KICK THEM WTF I DID NOT WANT TO BE SHOT AT.
I dunno, you guys have survived Thomas the tank engine, golems, zombies, nuclear explosions, laser whales, and being on the same team as ragnarock.  I don't think something as tame as a world ending rain of lava will even slow you guys down.

Googolplexed

  • Bay Watcher
  • My avatar is of whitespace, Not the firefox logo
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #3 on: April 10, 2010, 10:56:47 pm »

In theory yes.

But now, are you willing to write that program.
There's no way to automatically do what you asked for, the computer doesn't know which parts are part of the gameplay and which parts aren't, its an oversimplification, but, yeah.

« Last Edit: April 10, 2010, 11:01:44 pm by Googolplexed »
Logged

psyn

  • Bay Watcher
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #4 on: April 10, 2010, 11:21:35 pm »

Memory cannot be serialized, because memory is allocated on demand, each block sized to request, placement determined by OS. Structure sizes in memory vary by the second and between program loads -- it's not statically allocated. Offsets would need to be rewritten on write, structures resized (and all data after rewritten again), so you would need to know the exact memory layout of that program, impossible to do as a generic task. And there is no telling how it is going to react; a single old device handle will cause it to blow up. Essentially, if a program was not designed to serialize data, it cannot be made to.
Logged

Googolplexed

  • Bay Watcher
  • My avatar is of whitespace, Not the firefox logo
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #5 on: April 10, 2010, 11:50:24 pm »

Memory cannot be serialized, because memory is allocated on demand, each block sized to request, placement determined by OS. Structure sizes in memory vary by the second and between program loads -- it's not statically allocated. Offsets would need to be rewritten on write, structures resized (and all data after rewritten again), so you would need to know the exact memory layout of that program, impossible to do as a generic task. And there is no telling how it is going to react; a single old device handle will cause it to blow up. Essentially, if a program was not designed to serialize data, it cannot be made to.
Which is why I said on a program by program basis.
And yes I understated what a monumental task it would be.
Possible yes, practical no

(And a generic task is also possible, emulate the entire system and save the entire memory, VMware does that now)
« Last Edit: April 10, 2010, 11:53:38 pm by Googolplexed »
Logged

psyn

  • Bay Watcher
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #6 on: April 11, 2010, 12:09:19 am »

I was writing to Schilcote. :) Actually, I forgot the most important part: it would also need to rewrite the stack, an intrinsically undeterministic area of memory, by (1) its entirety or (2) the important sections. First is impossible because of stale recorded data, and #2 is impossible because of no way to identify it. I never heard of someone attempting this.

Cool thought about using a VM, but a snapshot of the system is needed, not just a lone program, no? I don't understand how it could work otherwise.
Logged

Schilcote

  • Bay Watcher
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #7 on: April 11, 2010, 06:53:39 am »

Hmm... what if I measured the changes in the program's allotted memory block, and only save memory values that change after the program is started?
Logged
WHY DID YOU HAVE ME KICK THEM WTF I DID NOT WANT TO BE SHOT AT.
I dunno, you guys have survived Thomas the tank engine, golems, zombies, nuclear explosions, laser whales, and being on the same team as ragnarock.  I don't think something as tame as a world ending rain of lava will even slow you guys down.

eerr

  • Bay Watcher
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #8 on: April 11, 2010, 08:50:47 pm »

what language was the program in?

not that it really matters, this is impossible without an emulator setup.
Logged

Blacken

  • Bay Watcher
  • Orange Polar Bear
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #9 on: April 12, 2010, 01:21:52 am »

I was writing to Schilcote. :) Actually, I forgot the most important part: it would also need to rewrite the stack, an intrinsically undeterministic area of memory, by (1) its entirety or (2) the important sections. First is impossible because of stale recorded data, and #2 is impossible because of no way to identify it. I never heard of someone attempting this.

Cool thought about using a VM, but a snapshot of the system is needed, not just a lone program, no? I don't understand how it could work otherwise.
There's no reason it's "impossible," although if you had said "impossible to generalize" I would agree with you. Same with spoofing file descriptors, subsuming mutexes, etc.--it's not impossible, just decidedly nontrivial, extremely failure-prone, and of little value. I remember reading about (the source escapes me at the moment) what was essentially the World's Most Batshit Replay Attack being constructed by rebuilding the memory space of a weakly random crypt program at a known point, then playing it forward. I'll check and see where I remembered seeing it.
Logged
"There's vermin fish, which fisherdwarves catch, and animal fish, which catch fisherdwarves." - Flame11235

Shades

  • Bay Watcher
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #10 on: April 12, 2010, 07:28:58 am »

Processes cannot be started from mapped memory.

That is actually all processes can be started from. But yes it would be excessively non-trivial to write a generic system. Depending on that particular game you might find it's doable but that would require you to investigate first.
Logged
Its like playing god with sentient legos. - They Got Leader
[Dwarf Fortress] plays like a dizzyingly complex hybrid of Dungeon Keeper and The Sims, if all your little people were manic-depressive alcoholics. - tv tropes
You don't use science to show that you're right, you use science to become right. - xkcd

psyn

  • Bay Watcher
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #11 on: April 12, 2010, 10:59:53 am »

There's no reason it's "impossible," although if you had said "impossible to generalize" I would agree with you. Same with spoofing file descriptors, subsuming mutexes, etc.--it's not impossible, just decidedly nontrivial, extremely failure-prone, and of little value. I remember reading about (the source escapes me at the moment) what was essentially the World's Most Batshit Replay Attack being constructed by rebuilding the memory space of a weakly random crypt program at a known point, then playing it forward. I'll check and see where I remembered seeing it.
OK, fair enough point. But we are talking about a game here. Meaning, probably at least several hundred MB's of allocated memory. I would rather try something 'easier' -- easier used very loosely -- like injecting an exception-catching DLL that hooks the relevant Win32 Create/Close functions, periodically saves program state, and attempts to rewind operation on crash.

BTW, I only just now realized (after typing that) that Schilcote was mentioning a PocketPC game. I still hold that this is next to impossible for any non-small game -- which would have thousands of open handles -- but OK, sure, you could more than likely do it for a program of this size. I hadn't even got to the matter of GPU or other hardware state and initialization.

But rebuilding all memory would still not address the problem of why it crashed. It might be restored, only to crash again a nanosecond later under the same circumstances. So to modify my previous statement, you can (1) restore all memory, or (2) restore precise game state. But #1 may be a futile effort, and #2 is impossible without reverse engineering the full game. Doing #2 would take 20 years for a AAA PC game that chaotically organized state in memory. It might be easier instead to simply solve the crash issue, or rewrite the game, or decompile/disassemble it for bug fixing and modification.

That is actually all processes can be started from. But yes it would be excessively non-trivial to write a generic system. Depending on that particular game you might find it's doable but that would require you to investigate first.
You're talking about how Windows works underneath. CreateProcess must be reimplemented to accept a handle to a buffer or memory-mapped file. I suppose you could write it to disk first, but that still isn't what I was getting at -- that you can't automagically resume program operation using dumped memory of the complete address space.
Logged

Shades

  • Bay Watcher
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #12 on: April 12, 2010, 11:17:07 am »

You're talking about how Windows works underneath. CreateProcess must be reimplemented to accept a handle to a buffer or memory-mapped file. I suppose you could write it to disk first, but that still isn't what I was getting at -- that you can't automagically resume program operation using dumped memory of the complete address space.

Actually I would have do something along the lines of how a system hibernates and resumes a process, as this is effectively the same thing we are talking about. The difference would be that once we have taken the snap shot for the hibernation we allow the process to continue instead of stopping it.
Logged
Its like playing god with sentient legos. - They Got Leader
[Dwarf Fortress] plays like a dizzyingly complex hybrid of Dungeon Keeper and The Sims, if all your little people were manic-depressive alcoholics. - tv tropes
You don't use science to show that you're right, you use science to become right. - xkcd

psyn

  • Bay Watcher
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #13 on: April 12, 2010, 11:30:12 am »

I know we're (over the course of this thread) discussing just that. I was responding to your sentence, "This is actually [how] all processes can [are?] be started from." Dunno, maybe I confused what you said.

Anyway, Raymond Chen mentioned this long ago. Why can't the system hybernate just one process? I haven't read it in years, but your post made me remember of it.
Logged

Blacken

  • Bay Watcher
  • Orange Polar Bear
    • View Profile
Re: A sort of a program memory backupy thing...
« Reply #14 on: April 12, 2010, 12:41:51 pm »

OK, fair enough point. But we are talking about a game here. Meaning, probably at least several hundred MB's of allocated memory. I would rather try something 'easier' -- easier used very loosely -- like injecting an exception-catching DLL that hooks the relevant Win32 Create/Close functions, periodically saves program state, and attempts to rewind operation on crash.
Personally, I'd rather just play a different game.
Logged
"There's vermin fish, which fisherdwarves catch, and animal fish, which catch fisherdwarves." - Flame11235
Pages: [1] 2