Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[hops]

It's not edgy misanthropy. If it's a bad idea to kill humans, then the AI won't kill humans. If it's a good idea, then it would.

[Execute/Dumbo.exe]

It's not edgy misanthropy. If it's a bad idea to kill humans, then the AI won't kill humans. If it's a good idea, then it would.

Well, at what point would it be a 'good idea'? We're giving this AI a set of morals, right? Or is it just building them itself? I can't really think of a situation in which any kind of moral code would intersect with reality to cause the AI to arrive at 'kill 'em all', and if it didn't have a moral code, then, well, it would exist in a vacuum, no action would be taken, unless you gave it aspects that made it enjoy certain things, which would naturally cause a moral code to arise, but even if it ended up enjoying murdering humans, then the hypothetical superintelligence would likely end up keeping humanity surviving, if anything, since it would always want more humans to kill.
See, what's 'right' is subjective, depending on who you ask, it could be a simple abstraction of morality (Don't kill things, try to help others, stuff like that) Or a weird bacon-and-bowtie set of internal rules that arise through a really insane mind.

Also, someone could be smarter than you but go with the wrong option, you have the accept the fact that even god could be wrong given a limited knowledge, same thing with a supercomputer, it may arrive at a conclusion that is entirely false based on wrong information, say I said that -1*-1=-1, I have just screwed up most of it's understanding of maths, now anyone with even a basic knowledge of multiplication could be better than the AI, and it would never know this unless someone else pointed it out and it felt that the second person was correct, it may arrive at an answer faster, certainly, but not the correct one.

[lemon10]

It's not edgy misanthropy. If it's a bad idea to kill humans, then the AI won't kill humans. If it's a good idea, then it would.

A super intelligence would do whatever it is evolved/designed to do very well. If we build an AI with the sole goal of making cheese (and forget to give it moral guidelines, because hey; its just a cheese AI), then it would kill all life on earth not necessary for making cheese to increase its cheese output, after which it would expand as fast as possible, converting every planet into dyson structures in order to increase yield.
Is that a good action? Unless you are the greatest cheese enthusiast that ever lived, you would probably say no.

[TempAcc]

Welp, a decent way to stop the "this AI made cheese out of the entire earth" scenario is giving the AI strict commands to only use the specifically designated materials, and also to tell it "hey, uh, please find out the best way to do this" instead of "hey, do this in the best way possible". This way you have an AI comming up with things in a controlling environment and then telling you what it discovered instead of just going around doing whatever it was commanded to.

We want the AI to come up with ideas and solutions for things, but not to implement said ideas and solutions by itself.

[Sergarr]

An even better idea is to, instead of a single AI deciding things, have multiple AIs made by multiple different competing companies cross-validate each other's actions.

[lemon10]

Welp, a decent way to stop the "this AI made cheese out of the entire earth" scenario is giving the AI strict commands to only use the specifically designated materials, and also to tell it "hey, uh, please find out the best way to do this" instead of "hey, do this in the best way possible". This way you have an AI comming up with things in a controlling environment and then telling you what it discovered instead of just going around doing whatever it was commanded to.

We want the AI to come up with ideas and solutions for things, but not to implement said ideas and solutions by itself.

The problem here is is in the first generation of true self improving AI.
After the first generation it will simply be impossible for a single hostile AI to do stuff like take over the internet, or create nanobots that eat the world for the simple reason that there will be other equal intelligences to stop it.

But unlike the other AI problems, this one is tricky in that if you fail to get it right the AI doesn't stop working or have any noticeable problems until it decides to start doing stuff (eg. mass cheese production) we really don't want it to do.
Its especially nasty because it only requires a single early group to fail and not take the proper precautions.

[martinuzz]

An even better idea is to, instead of a single AI deciding things, have multiple AIs made by multiple different competing companies cross-validate each other's actions.

The first two things that come to my mind here are "AI wars" and "caught in the crossfire"

[Gentlefish]

Welp, a decent way to stop the "this AI made cheese out of the entire earth" scenario is giving the AI strict commands to only use the specifically designated materials, and also to tell it "hey, uh, please find out the best way to do this" instead of "hey, do this in the best way possible". This way you have an AI comming up with things in a controlling environment and then telling you what it discovered instead of just going around doing whatever it was commanded to.

We want the AI to come up with ideas and solutions for things, but not to implement said ideas and solutions by itself.

The problem here is is in the first generation of true self improving AI.
After the first generation it will simply be impossible for a single hostile AI to do stuff like take over the internet, or create nanobots that eat the world for the simple reason that there will be other equal intelligences to stop it.

But unlike the other AI problems, this one is tricky in that if you fail to get it right the AI doesn't stop working or have any noticeable problems until it decides to start doing stuff (eg. mass cheese production) we really don't want it to do.
Its especially nasty because it only requires a single early group to fail and not take the proper precautions.

I'm surprised this bother you, considering all early test AI's will be chained to an outlet for power consumption. Cut the power, and poof! No more mean bad AI.

[forsaken1111]

Look nobody said human worries are rational. I still worry about the ceiling fan falling on me in hotels

[Execute/Dumbo.exe]

Look nobody said human worries are rational. I still worry about the ceiling fan falling on me in hotels

Well, yeah, that's why if I was actually researching something that dangerous I would be hooking up so many failsafes and failsafes for failsafes in one big Faraday cage, on a close circuit system, in Antarctica, that I would purposely forget a few just so if hypothetically it convinced me to let it go, it would still be turned into a pile of binary and bitcoins.

[Dozebôm Lolumzalìs]

I think there's reason to be afraid of making something that thinks like we do yet is qualitatively better at thinking than we are, you can't fully comprehend the mind of something more intelligent than you, it's like asking a computer to emulate a supercomputer, your emulation is flawed from the outset.

I find this analogy flawed. As a Turing machine, given sufficient memory, a computer can emulate any other computer. The AI-ness is inherent in the code, not the computer.

[Max™]

Yep. It could emulate one, it just couldn't do so terribly fast due to the limitations of the hardware.

Yup, I am qualitatively better at thinking than my cat, I can hold a model of his brain in my mind and suffer no significant slowdown, he is unable to begin trying to model my brain, but he can model a mole brain damn well, hence his success at ridding us of them.

[Reelya]

It's a little different with computation devices.

A ridiculously simple CPU (far simpler even than what's in a cheap calculator) could model the most complex computer on the planet, given enough RAM. It wouldn't be fast, but it would in fact be a perfect digital copy of the supercomputer. And this isn't just conjecture: there are rigorous mathematical proofs that this is the case. Which is science².

If meat brains worked the way computation does, then a mouse could out-think Einstein, given enough writing paper to jot things down on.

[Dozebôm Lolumzalìs]

Yeah, physical constraint's aren't the problem. It's that we don't have code for AIs. Remember Hex?

rip pratchett

[scrdest]

I bet you can train a mouse to act like a Turing machine. Therefore, it's emulating a computer which can emulate the most complex computer with the right set of instructions. Ergo, a mouse can out-think Einstein.

A bet on a conjecture is not a proof.