Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[miauw62]

You don't have to generate them all at once.
I'm completely ignorant on the exact implementation, but infinite-world games usually use a seed and generate things on the fly.

[EnigmaticHat]

Sorry, yeah, it doesn't.  It generates each planet as you click on it.  Right now it holds all the maps in memory so at a certain point that would be a problem but it would be pretty easy to pickle them or add their info to a text file or whatever.  Generating them all (even one at a time over a long period of time) and having AI civilizations colonizing them and stuff could still be a source of slowdown.  The main problem is drawing them.  Also the fact that sometimes when I generate a planet memory usage goes sky high and I get a "memoryError" (that's the entire message), but that doesn't matter since I'm porting it over anyway and I'm going to try to make it more efficient when I do.

But I'm not really seeking help here, I was just chiming in on the whole "Python can be slow" thing.  Figured I'd show off my maps since people kind of asked and I'm pretty proud of them.

Spoiler: here's some more (click to show/hide)

Also the crappy placeholder solar systems

And the crappy placeholder galaxy map

[miauw62]

That does look pretty nice.

[DJ]

How can you run out of memory? Are you allocating everything on the stack or something?

[ed boy]

I'm back with another C question, I'm afraid.

This time, I'm having issues with error handling. As far as I can tell, the only error handling that exists in C is longjmp and setjmp. As I understand it, using these involves setting a program state, then later giving you the option to return to that program state, One would use this to handle errors by checking if the inputs for a function are valid before calling it, and restoring the program state that was chosen originally. The use of this method requires you to be able to determine if the inputs for a function are valid without actually calling it.

In my situation, I am attempting to read values from a csv as numbers, do some math and put the result into another csv. Unfortunately, some of the files contain values in the entry that cannot be read properly. This causes errors. Here's the relevant snippet of the code:

Code: [Select]

        ptr_pricelist = fopen(pricepath,"r");
        fgets(buf,1024,ptr_pricelist); //load up the header of the file
        fgets(buf,1024,ptr_pricelist); //look past the header to the second line
        //We want the values in the 5th column
        strtok(buf,",");
        strtok(NULL,",");
        strtok(NULL,",");
        strtok(NULL,",");
        strcpy(value,strtok(NULL,","));           //This is where the errors happen
        oldprice=atof(value);
Is there any way handle these errors? I can't use the longjmp/setjmp method, because I don't know how to check whether the code in question will cause errors without actually running it.

[Reelya]

In what way can't the data be read? Are you checking that the value returned is non-NULL? Maybe you can manually check if the entry is a number or not.

[ed boy]

Are you checking that the value returned is non-NULL? Maybe you can manually check if the entry is a number or not.

Thanks, that worked. I forgot about checking for null pointers, and I was thinking that since I don't know the inner workings of the atof function, I don't know what the requirements of its inputs are.

I have had a new problem, however. The relevant part of code is:

Code: [Select]

        value=strtok(NULL,",");
        if (value==NULL)
        {
            oldprice=0.0;
            isinvalid[i]=1;
        }
        else
        {
            oldprice=strtod(value,NULL);
            printf("%d %s %f\n",i,value,oldprice);
        }
The problem is that the variable oldprice isn't assigning properly. The printf statement tells me that although value is being picked up correctly the variable oldprice is incorrect - it becomes a large integer, which I (perhaps naively) believe to be a pointer. This incorrect value of the variable oldprice is the same no matter what i or value is.

EDIT: I figured it out. It turns out that I forgot an include at the start of my file, yet my compiler compiled everything anyway.

[EnigmaticHat]

How can you run out of memory? Are you allocating everything on the stack or something?

I have no idea, it just shoots through the damn roof.

I've watched it in task manager.  The program's memory use increases a small amount for each map generated, totaling less than even the IDE uses after a few planets.  Then it'll freeze and the memory use listed in task manager will rise and rise and rise until it hits some specific limit, and then the console will write "memoryError" and the process will stop responding.

[Antsan]

I really have no idea what I am talking about, but I think that may be an endless recursion somewhere or maybe a loop allocating stuff which is not terminating properly.

[Orange Wizard]

Try setting some breakpoints and see if you can't narrow down where the memory bug is spawning.

[monkey]

Check it with https://pypi.python.org/pypi/memory_profiler

Nice looking maps btw.

[ed boy]

I'm afraid I'm back with another problem in my adventures with C.

My current problem is that I have a large number of files, each containing an ordered list of integers. I wish to construct a list that contains each unique integer from the set of files. Normally I would approach this by using a modified merge sort algorithm, but I have a problem - the number of files and the length of each file are sufficiently large that loading them all into memory would be impractical. My next thought would be to iteratively merge sort the files, but that would require me to either define an array big enough to hold the worst-case scenario (which is impratical), or to write a loop that, on each iteration, defines an array large enough to hold the newly merged list, which is beyond my C capabilities. Is there a better way to approach this problem, or can someone help me with writing a loop that can do this?

[Twiggie]

Why is defining an array big enough to hold the worst case impractical? It's only 500MiB, no?

Use a bit field of size uint.max and set the appropriate values as you iterate over. O(n) unless I'm mistaken?

E: though I suppose that doesn't make use of the fact that the files' contents are sorted.

[MagmaMcFry]

I think they want you to approach this problem like follows:

Make a list of input streams, one for each file. Now make another list of integers, of the same size, and fill it with the first number in each file. Now pick the smallest of those numbers, output it, and replace all instances of that number in your list with the next number from each corresponding stream. Repeat until all files are finished, then close your streams.

[ed boy]

Why is defining an array big enough to hold the worst case impractical? It's only 500MiB, no?

The number of files can be anywhere from two thousand to ten thousand, and the number of integers per file could be anywhere from one to thirty thousand, and those numbers could increase in the future. That's potentially three hundred million integers, and that number could go up in the future.

Make a list of input streams, one for each file. Now make another list of integers, of the same size, and fill it with the first number in each file. Now pick the smallest of those numbers, output it, and replace all instances of that number in your list with the next number from each corresponding stream. Repeat until all files are finished, then close your streams.

how efficient is it to have thousands of input streams open at the same time?