Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2

Author Topic: ChatGPT to optimize source code?  (Read 2107 times)

Streetguru

  • Escaped Lunatic
    • View Profile
ChatGPT to optimize source code?
« on: April 07, 2023, 06:27:54 am »

Hi, just wondering if ChatGPT has been played with at all by the devs to potentially optimize the source code of the game?

It was able to find at least one optimization for Super Mario 64 here:

https://www.youtube.com/watch?v=9QZlQMpNk-M

I've heard GPT4 is like way better though than the version on the main website as it stands. Would require a premium subscription for that version.
Logged

delphonso

  • Bay Watcher
  • menaces with spikes of pine
    • View Profile
Re: ChatGPT to optimize source code?
« Reply #1 on: April 07, 2023, 06:54:08 am »

I wouldn't be surprised if Putnam is doing a bit of this, as it seems that optimization is their main focus.

It's useful to think of chat AI as just the next step of Search. In this video as well, you can see this guy is basically just googling, but with the added layer of some memory and clarifying questions, plus real-time interpretation of code.

GPT4 has promise for whole-cloth editing code. I don't think we're that far away from just copying and pasting a codebase into an AI and getting an optimized version spat out, but there will always be limits, and DF is certainly a formidable codebase.

Buttery_Mess

  • Bay Watcher
  • 11x11
    • View Profile
Re: ChatGPT to optimize source code?
« Reply #2 on: April 07, 2023, 08:17:53 am »

ChatGPT would almost certainly just break the code and introduce many bugs into it. DF's code is unusually long and couldn't be included in a single prompt.
Logged
But .... It's so small!
It's not the size of the pick that counts... it's the size of the man with the pick.
Quote from: Toady One
Naturally, we'd like to make life miserable for everybody, randomly, but that'll take some doing.

Streetguru

  • Escaped Lunatic
    • View Profile
Re: ChatGPT to optimize source code?
« Reply #3 on: April 07, 2023, 10:35:07 pm »

ChatGPT would almost certainly just break the code and introduce many bugs into it. DF's code is unusually long and couldn't be included in a single prompt.

You could just do it one section at a time, or point it to specific examples. Not sure if the GPT4/Plus version has more flexibility for input.
Logged

delphonso

  • Bay Watcher
  • menaces with spikes of pine
    • View Profile
Re: ChatGPT to optimize source code?
« Reply #4 on: April 07, 2023, 10:47:01 pm »

They do, but natural language processing does have issues with understanding loops, just from my own experience. GPT4 may have overcome some of that, but certainly the ones to come could help out.

Also, just for the human-side of things, you could really only handle one or two sections of the code at a time. Assuming you want to understand the changes made, there's really a lot that could be 'fixed' in any bit of code.

Putnam

  • Bay Watcher
  • DAT WIZARD
    • View Profile
Re: ChatGPT to optimize source code?
« Reply #5 on: April 08, 2023, 04:03:38 am »

I wouldn't be surprised if Putnam is doing a bit of this, as it seems that optimization is their main focus.

It's useful to think of chat AI as just the next step of Search. In this video as well, you can see this guy is basically just googling, but with the added layer of some memory and clarifying questions, plus real-time interpretation of code.

GPT4 has promise for whole-cloth editing code. I don't think we're that far away from just copying and pasting a codebase into an AI and getting an optimized version spat out, but there will always be limits, and DF is certainly a formidable codebase.

Chat-GPT is pretty much worthless for anything of moderate complexity and especially anything unique; I haven't considered using them and I am not likely to in the future, since DF is generally both of those in spades. I would be genuinely surprised if GPT-4 were a major improvement

Ziusudra

  • Bay Watcher
    • View Profile
Re: ChatGPT to optimize source code?
« Reply #6 on: April 08, 2023, 02:36:45 pm »

There's also the concern that what you share with ChatGPT can be stored on multiple someone else's high interest computers:
We share content with a select group of trusted service providers that help us provide our services
and to a service that has already been tricked into doing things it wasn't supposed to.
« Last Edit: April 08, 2023, 02:38:35 pm by Ziusudra »
Logged
Ironblood didn't use an axe because he needed it. He used it to be kind. And right now he wasn't being kind.

Cathar

  • Bay Watcher
  • Competent Engraver
    • View Profile
    • My shit
Re: ChatGPT to optimize source code?
« Reply #7 on: April 08, 2023, 02:43:22 pm »

Using ChatGPT to code/optimize is a receipe for disaster. ChatGPT is correct most of the time, but it also "guesses" and makes stuff up, and has no way to distinguish between genuine information provided and information created from various cloth.
It's a great tool to learn method, to get ideas, possibly to analyze code, but if you ever tried to blidnly copy-paste an even moderatly complex chatgpt-generated code into Visual Studio you immediately know why it's a bad idea.

ChatGPT is a great tool, but it's nowhere near good enough to completely replace human coder or act without strict supervision.

LuuBluum

  • Bay Watcher
    • View Profile
Re: ChatGPT to optimize source code?
« Reply #8 on: April 08, 2023, 03:06:49 pm »

Not to mention it falls apart the more "specialized" or "built-up" the working environment. It's just guessing at names and all the rest of it based on statistical likelihood, so it's not like it can magically discern what any data structure actually holds. Only what it assumes it holds given whatever was provided.

As for using it to analyze code? We have far better tools already written to that end.

I'm still waiting for an actual use case for these things, besides writing for Buzzfeed.
Logged

Cathar

  • Bay Watcher
  • Competent Engraver
    • View Profile
    • My shit
Re: ChatGPT to optimize source code?
« Reply #9 on: April 09, 2023, 08:05:24 am »

I'm actually using ChatGPT to learn python. It is really a great tool - for what it is. AI looks very impressive at first glance, but it has inherent (and possibly unsolvable) issues. When the guys from AI security say "there is no algorythm for truth", they mean it. ChatGPT is totally blind, it doesn't "knows" what it is doing and acts according to a very complex mechanism of rewards and punishments.

For exemple, I asked it to fetch the endpoint of an API for a french marketplace - it could not find it, so he "invented" an endpoint by mixing the french version of the website and the japanese endpoint. Of course it doesn't work - but for one it is very hard to notice the flaw because without testing the script it looks correct, and two the AI does not make the difference between a factual endpoint and the one it just pulled of its ass.

Asking an AI to smooth up your code is the best way to have it pop unpredictably at places you can not expect it to pop.

Still ChatGPT is a great tool to learn. It saves me a lot of time and a lord of headaches asking questions to actual programmers on forums.
« Last Edit: April 09, 2023, 08:08:24 am by Cathar »
Logged

Putnam

  • Bay Watcher
  • DAT WIZARD
    • View Profile
Re: ChatGPT to optimize source code?
« Reply #10 on: April 09, 2023, 03:04:45 pm »

ChatGPT is correct most of the time

I've actually never gotten it to be correct once, since I keep asking it intermediate-level questions, which it fails at consistently.

Like, some examples:

Quote from: me
There was a thought experiment about a computed being who uses less and less energy over time, thus slowing down, but never perceives the slowdown, thus lasting forever. Who came up with this?

Quote from: Chat-GPT
The thought experiment you are referring to is known as the "Ultimate Fate of the Universe," and it has been discussed by various scientists and philosophers over the years. One of the earliest recorded discussions of the idea is attributed to the British mathematician and philosopher William Kingdon Clifford, who wrote about the possibility of an "intelligence which, at some indefinitely remote period, shall have learned to keep itself alive indefinitely, by constantly occupying itself with some minute and unchanging action" in an 1875 essay titled "The Ethics of Belief."

The idea was also explored by the American mathematician and logician, Bertrand Russell, in his 1923 book "The Analysis of Mind." In the book, Russell wrote about the possibility of a being that could exist for an infinite amount of time by gradually slowing down its thought processes, thus reducing its energy consumption, without ever perceiving the slowdown. The idea has since been further developed by other thinkers in fields such as physics, computer science, and philosophy.

This put me down a minor rabbit hole of "that can't be right..." until someone pointed out that my first guess (Freeman Dyson) probably did it, and yeah, Freeman Dyson introduced the concept in 1979. This isn't novel stuff, it's just mildly obscure, and Chat-GPT failed utterly at it. And I have tons of this, actually.

Quote from: me
What is the difference between std::execution::unseq and std::execution::par_unseq?

Quote from: Chat-GPT
std::execution::unseq specifies that the algorithm should be executed in an unordered, potentially parallel manner. This means that the algorithm can be executed in any order, and may be executed in parallel if it is more efficient to do so.

That "may be executed in parallel if it is more efficient to do so" bit is confabulated; the spec doesn't say this and in fact implies the precise opposite.

The closest I've seen it get to correct was the following exchange:

Quote from: me
What is the *n* that results in the largest *n*-ball of radius 2?

Quote from: Chat-GPT
The largest $n$-ball of radius 2 is the $n$-dimensional sphere of radius 2, denoted as $S^n(2)$.

The volume of an $n$-dimensional sphere of radius $r$ is given by the formula:

$V_n(r) = \frac{\pi^{\frac{n}{2}}}{\Gamma(\frac{n}{2} + 1)}r^n$

where $\Gamma$ is the gamma function.

Therefore, the volume of the $n$-dimensional sphere of radius 2 is given by:

$V_n(2) = \frac{\pi^{\frac{n}{2}}}{\Gamma(\frac{n}{2} + 1)}2^n$

To find the value of $n$ that maximizes $V_n(2)$, we can take the derivative of $V_n(2)$ with respect to $n$ and set it equal to zero:

$\frac{dV_n(2)}{dn} = \frac{\pi^{\frac{n}{2}}}{2\Gamma(\frac{n}{2} + 1)}(n\ln 2 + 2\ln \pi - \psi(\frac{n}{2} + 1)) = 0$

where $\psi$ is the digamma function.

Solving for $n$ numerically, we find that the value of $n$ that maximizes $V_n(2)$ is approximately 5.257.

Therefore, the $n$-ball of radius 2 with the largest volume is the 5-dimensional ball of radius 2.

Its math is perfectly valid if you take n=1, but that's not what I asked.

I haven't gotten it to say anything that's actually completely correct about anything. Since correctness is always the #1 priority in programming, it's a terrible idea to use it for that.
« Last Edit: April 09, 2023, 03:15:32 pm by Putnam »
Logged

Cathar

  • Bay Watcher
  • Competent Engraver
    • View Profile
    • My shit
Re: ChatGPT to optimize source code?
« Reply #11 on: April 09, 2023, 04:48:00 pm »

Well, with the questions I ask it for, it is correct most of the time. But then again I ask it basic questions about skills I'm learning for the purpous of widening my own knowledge, not testing the AI. "What does that error means ?" "What course of action do you suggest to fix it ?" etc. It's like having an expert in almost any field at arms lenght, and in my case, it saves me to have to hire a consultant.
Are you the one who did try to generate Dwarf Fortress Raws with AI, and it also ended up making stuff up completely ?

AI technology is nowhere near mature enough to work on DF

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: ChatGPT to optimize source code?
« Reply #12 on: April 09, 2023, 05:53:41 pm »

Are you the one who did try to generate Dwarf Fortress Raws with AI, and it also ended up making stuff up completely ?

Ask ChatGPT. ;D
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

Cathar

  • Bay Watcher
  • Competent Engraver
    • View Profile
    • My shit
Re: ChatGPT to optimize source code?
« Reply #13 on: April 09, 2023, 07:24:10 pm »

Alas, it cannot access databases and only know what it has been Pre-Trained with (the PT in GPT).
I could just check the forum but over-reliance on AI technology made me lazy

A_Curious_Cat

  • Bay Watcher
    • View Profile
Re: ChatGPT to optimize source code?
« Reply #14 on: April 09, 2023, 07:33:04 pm »

Alas, it cannot access databases and only know what it has been Pre-Trained with (the PT in GPT).
I could just check the forum but over-reliance on AI technology made me lazy

A few seconds of searching the modding subforum for the string ‘AI’ revealed that it was Xrazlyn and not Putnam.
Logged
Really hoping somebody puts this in their signature.
Pages: [1] 2