Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[i2amroy]

CLion master race!

It's a newer one made by the same people who did IntelliJ, which I love, only difficult part is getting a valid license for it (I'm using my C:DDA open source license ATM).

[BlindKitty]

Try adding more print statements to check the values of variables during the run. e.g. each loop print x and book[ x ] with this: --snip--

You can also try using a language that actually gives you decent error messages and/or decent debugger with Watch option (which just keeps showing you the value of a variable without printing it all the time). (That is, of course, unless you need C/C++ for some other reason).

C++ also happens to have the best debugger ever, aka Visual Studio!

Well, I'm using the Visual Studio with C#, so that was actually what I had in mind; but I don't know anything about using C++ with Visual Studio. Aren't you locked into Microsoft's own compiler/managed C++ then? I honestly don't know.

That being said, I see no reason to stick to C++ if you don't need it for some highly specific reason*. It's relatively hard to write, hard to debug, and doesn't offer anything other languages don't offer. Even C# is going cross-platform nowadays.

*Extreme speed may be such reason, but there are honestly fewer and fewer applications where that does matter, and even in those, you are probably better of with C.

[Reelya]

No, you're not locked into any sort of managed code. You can write cross-platform code in MSVC

BTW I recommend any people thinking about c++ to take a look at Mike Acton's videos. Basically, if you take cache reads into account you can get substantial speedups (like 5-10 times faster) on even "trivial" looking code. Relevant to C++ and probably helps in other languages too.

https://www.youtube.com/watch?v=GPpD4BBtA1Y

This video below covers a lot of the same as the above video, but is longer with extra stuff. Basically if you get rid of the "object" idea and treat everything as data streams that need to be processed, and each stream only contains the data that you're actually processing, you can get the order-of-magnitude speedups in your code. That's because each time you read a value from memory it reads an entire cache line (e.g. ~64 bytes) into the cache. If that block of memory include a lot of shit you're not actually using in your algorithm, then it's basically wasted memory reads, and raw memory reads are extremely slow, ~200 clock cycles, vs reading the next value from L1 cache, which takes ~3 clock cycles.

https://www.youtube.com/watch?v=rX0ItVEVjHc

[Telgin]

Those sound really interesting and I'll have to give them a look.  I always assumed that compiler optimizations would make it too difficult to predict memory access patterns for nontrivial programs, so I rarely bother trying to hand optimize code for cache access improvements.  Pretty much the only rule that I stick to there is to not iterate nested loops with the indices reversed.  That is, don't do this:

Code: [Select]

// some_array is a 2D array already initialized
for (int i = 0; i < Y_SIZE; i++) {
    for (int j = 0; j < X_SIZE; j++) {
        some_array[j][i]++;
    }
}

Depending on how some_array is built that may be the right way to iterate it, but in most cases that's going to cause memory reads to be very inefficient because it's going to be jumping around the address space by the width of the array lines with every access.

[Reelya]

Mike's first example in video #1 is that exact example you printed.

Mike's main point is that it's retarded to rely on this magic sauce of "compiler optimization" because it's actually pretty limited. The compiler isn't going to change the actual memory layout that your specify in any way whatsoever, so we can in fact reason extremely precisely about hand-optimizing memory packing of the data we are reading.

if you have an array of 64-byte structs and you're reading these in a loop to get 1 value per struct then you are in fact reading an entire 64-byte cache line per single value that you want. Say that you were reading a bool from each 64-byte struct calleds "b_needsProcessing" and only doing the processing if it's true. That's ~200 clock cycles to look up a single bool, and you haven't even done any work yet. If only a small number of items need processing on any specific tick that's a lot of wasted time right there. You can do a couple of other things - make an array of just bools called "needs processing" which maps to the objects. Then read that and only process the correct ones. You now got it down from 200 clock cycles per bool check to about 3 clock cycles per bool check. Mike talks about the importance of printing out your data to see what's happening. For example here, if only a small number of elements in the array actual need processing on any particular iteraction, it's doubly-wasteful to check the "needs_processing" bool for every object each frame at all. You can just make an array holding the ones that need processing on the next iteration, and cycle through that array instead of having the bool-check if-statements at all, which means less memory access time and better lookahead due to branching.

another mike example:

for(my loop)
{
    if(m_bool)
    {
        do_the_thing()
    }
}

vs

if(m_bool)
{
   for(my loop)
   {
      do_the_thing()
   }
}

A compiler such MSVC can't actually optimize version 1 into version 2. It can't do the optimization that you can see immediately of moving the bool check outside the loop. That's because you can reason non-locally that the member bool isn't changing as the loop progresses. But what if did change inside "do_the_thing"? Is the compiler meant to read your entire program and reason about such things, and make sure there aren't any concurrent processes that might have changed the m_bool value during the for loop? This would be non-trivial AI level stuff that doesn't actually exist.

[Telgin]

With compiler optimization I usually just assume that it's going to simplify or reorder operations, mostly to hide latency in the CPU's pipeline.  Thinking on it though, that probably pretty rarely affects memory access patterns.  It could if some part of an arithmetic operation were simplified, perhaps, but the overall access pattern is probably going to be more dominated by the layout and loops in the program.

I probably just need to watch the videos since I'm having a hard time imagining ways that you can really improve that for most programs, at least not without completely restructuring them.  Kind of like the getting rid of objects idea thing you mentioned.  Most of the time you'd have an array of complex objects that you iterate and access member variables from, but presumably, depending on what your code did, you could instead just maintain arrays of each member variable and iterate those instead.  That could be a lot faster but would probably be a much bigger headache to manage...

I guess that's kind of the eternal tradeoff with programming languages.  Do you want easy to use, or fast?  Rarely do you have both.

A compiler such MSVC can't actually optimize version 1 into version 2. It can't do the optimization that you can see immediately of moving the bool check outside the loop. That's because you can reason non-locally that the member bool isn't changing as the loop progresses. But what if did change inside "do_the_thing"? Is the compiler meant to read your entire program and reason about such things, and make sure there aren't any concurrent processes that might have changed the m_bool value during the for loop? This would be non-trivial AI level stuff that doesn't actually exist.

This is an important statement and something to think about, but it's worth noting that depending on how your code is structured it might indeed be able to do that optimization.  C and C++ compilers are usually pretty lazy about it or have a separate option for "global program optimization", but if it does have that option and it's turned on, they could possibly inline the do_the_thing function and perform code analysis to determine if the if check can be hoisted out of the loop.  I wouldn't rely on it though.

This is related to something I found surprising at first, which is that Fortran programs are generally considered faster than C programs, and that's because the compiler is usually able to optimize them better due to the language having stricter rules on memory usage.  In particular, pointers in C and C++ usually mean that the compiler has to do away with some optimizations because it can never know for sure if a variable is being aliased by several pointers that might change its value in unexpected ways.  Using the "restrict" keyword can be a useful hand optimization in such cases since it's effectively promising to the compiler that you're not doing that.

Now I'm curious if compilers even think about the possibility of nonlocal thread access to variables when performing optimizations.  I'm guessing not, unless you reference a variable through a pointer somewhere.

[Bumber]

My final assignment in my assembly programming course was to write an assembly function that messed with the variables and return addresses of the function chain that called it. I feel somewhat dirty, yet also like some kind of leet haxor.

[Vicomt]

not sure whether I should cross-post this to the WTF thread, but people there possibly wouldn't understand it.

I just found a hard-coded reference to the user named "admin" in our (released) software that rather large financial institutions use to manage cash.

heads are gonna roll for this. I quote our dev manager from one of his emails on the subject....

May I compliment both of you (not sure who actually did it) on some true GCSE level coding:

        //add the admin user to the log in link
        CommonPart MenuLinkCommonPartForLinkBar = MenuLinkForLinkBar.As<CommonPart>();
        var adminUser = contentManager.Query<UserPart, UserPartRecord>().Where(u => u.UserName == "admin").List().FirstOrDefault();
        MenuLinkCommonPartForLinkBar.Owner = contentManager.Get<IUser>(adminUser.Id);

So, providing our admin user is always called ‘admin’ we are good. It’s a good job this product isn’t live in the field….

W.T.F.

[TheBiggerFish]

... What the whatting what WHY.

[itisnotlogical]

So you can't have an admin user that isn't named "admin" or else it won't create some vital interface element?

[Pseudo]

So, I read an article.

It decompiles java class files. And then uses regexes to "parse" the output:

Code: [Select]

regex = re.compile(".*class [a-zA-Z0-9]+ extends Applet")
regexScript = re.compile(".*javax.script.ScriptEngine")
regexReflection = re.compile(".*java.lang.reflect")
Facepalms galore. (Note that, among other things, Java class names are general unicode, not just [a-zA-Z0-9]+ (for instance, _ is valid. Or αρετη). And one can use reflection without actually triggering that regex (among other ways, by embedding an obfuscated class file and loading it at runtime). And those regexes use . as a literal - which means that something like "java_lang_reflect" would also trigger it! Or "java.lang.reflect" in a string or comment(!)...

Oh, and the cherry on top is that the decompiler it uses executes (parts of) class files to try to better decompile them, and doesn't sanitize said input properly (allowing for java.lang.Runtime.getRuntime().exec(), which, if you don't know, provides shell access!) (which is a facepalm in and of itself)

(And there's also a root exploit, which doesn't surprise me given the rest of it.)

[Vicomt]

So you can't have an admin user that isn't named "admin" or else it won't create some vital interface element?

we use the Orchard CMS, part of it's setup (cooking the recipe... lol) is constructing UI elements, which failed when I was setting the thing up on my dev machine. I tracked the failure down to this code, The dev in question just used a query to get the admin user to assign to the owner of the element. Doing it via a hard-coded "admin" user name though is just o_O

[TheBiggerFish]

Well that is a biiiiig security flaw...

With an appropriate number too.

[Putnam]

So, I read an article.

It decompiles java class files. And then uses regexes to "parse" the output:

Code: [Select]

regex = re.compile(".*class [a-zA-Z0-9]+ extends Applet")
regexScript = re.compile(".*javax.script.ScriptEngine")
regexReflection = re.compile(".*java.lang.reflect")
Facepalms galore. (Note that, among other things, Java class names are general unicode, not just [a-zA-Z0-9]+ (for instance, _ is valid. Or αρετη). And one can use reflection without actually triggering that regex (among other ways, by embedding an obfuscated class file and loading it at runtime). And those regexes use . as a literal - which means that something like "java_lang_reflect" would also trigger it! Or "java.lang.reflect" in a string or comment(!)...

Oh, and the cherry on top is that the decompiler it uses executes (parts of) class files to try to better decompile them, and doesn't sanitize said input properly (allowing for java.lang.Runtime.getRuntime().exec(), which, if you don't know, provides shell access!) (which is a facepalm in and of itself)

(And there's also a root exploit, which doesn't surprise me given the rest of it.)

is java even regular

[Mephisto]

is java even regular

Generalizations and all that but no programming languages are regular.