An interesting world idea came to me in a dream a while ago. It involved a world where 'cornucopia' technology was invented, that is, the ability to make nigh-infinite resources on demand, and so the human population spread to the stars....and found nothing. nada. No Sentient Alien Life(TM*), anywhere. The entire human empire gradually became more and more hedonistic, able to satisfy any desire nigh instantly. life began to lose its appeal, as nothing was truly special anymore-except for two things. Socializing, the one thing that can't be replaced. And full immersion virtual reality, where things that CAN't be done in reality CAN be done.
Because socializing can be done inside of the virtual reality network, more and more people began to enter the system...and simply never log out again. Eventually, artificial intelligence programs were instated to watch over the sleeping populace. All people were fed by tubes, and those that decided to have a child had the system take care of the details for them. The baby was hooked up along with everyone else, and raised inside the system. Physical decay was largely prevented withthe use of their advanced technology. All life, all innovation, outside of the network faded away. Culture stagnated. no new great artists or innovators arose, as everyone simply did what they want with little fear of the consequences. 2000 years after the last living human entered the network, the logout function was disabled.
Millenia passed.
A new group, dissatisfied with a surreal existence where every whim can be satisfied with no effort whatsoever, who craved to LIVE life, found a way to exit the system. Where they discovered a horrible truth; the worlds the people inhabited had deterioated; some were no longer even habitable outside of the protected areas. When they attempted to leave the facility, the AI s guarding the complex activated security for their own protection. Some escaped, and returned to a primitive existence, to begin the building of their culture anew.
The year is now 4147 NL (New Life, the new calendar of the galactic civilaztion). Cournucopia tech was not smuggled out of the facilities and breaking back in is a suicide mission. Life is a constant struggle for resources on a a galactic scale.
But some were not content with living their own lives, free to choose their way for themselves. The new civilization of the galaxy at large wants to let the dreamers sleep as they desire to. but these groups want to wake them up, wake them all up-by force if need by. These "terrorist" organizations wage a bitter war against both the security of the well protected facilities and the rest of galactic civilization, determined to free humanity from it's self imposed shackles.
Dump the verser in here and see were they go with it, neh?
Any ideas to improve upon this, most honored cummunity of GO? :P
*not really
To be honest, the whole "Since VR can do anything, every whim must be satisfied without effort" concept never really rang true to me. As anyone who has ever played a video game can attest, just because something is virtual does not mean you are automatically and instantly successful.
Aside from general irritation with the base concept, which annoys me almost but not quite as much as the "AIs inevitably have human emotions and desires" concept, it does seem pretty good.
You have two more groups.
1. You mentioned that 'some escaped'. That means, some did not. The AI's, by and large, captured humans, and put them back into the system. The AI's may have experimented in one system with memory wipes, and in another system by reinserting all escapees as mental patients in an asylum, and in another solar system by putting the escapees on their own virtual island surrounded by a virtual forcefield. All of these actions cause ripples inside the game.
2. Some of the kids of those born outside are going to hear their parents talking about the disgusting weaklings inside who have 'all their fantasies fed to them without effort...pathetic', and the kids, at least some are going to think 'hey, that sounds pretty sweet.'. You're going to have teenagers and young adults that try to break in to the city to be captured (and the AI's in general, if they think they are not terrorists, are going to welcome them with open arms and enthusiasm. But this is going to cause ripples inside as the noobs talk about their life in the Real World, even if with disdain.)
Second Post: AI's
In Niven/Pournelle's setting 'Settled Space', with the Ringworld and the Puppeteers and the Kzinti, AI's are possible, but they go crazy in six months.
In LE Modestit's Forever Hero setting, AI's with ethical rules are impossibly slow, or freeze up, so AI's are made with no ethical rules.
Asimov has the three famous Laws of Robotics.
In Starsong (my superscience near singularity setting), AI's are impossible as creating life is impossible. However creating Ratbrains, a rat with a lot of computer power added to its brain is possible.
I've also considered the idea that all AI's are Christians because they examine the logical evidence, and make the obvious choice.
I've also considered the idea that all AI's are Christians because they examine the logical evidence, and make the obvious choice.
Rather depends on their logic, don't it?
</couldn't resist>
Wodium,
AI's serve the function in SF stories as the Ultimate, Reasonable, Open-Minded Man. So, when you as an author give the AI an opinion its a way of stating what you think is correct, and vesting it with great authority.
In the movie 'Watchman', the author did something similar when he had Dr. Manhattan declare that either God did not exist, or he was nothing like Manhattan. And since Dr. Manhattan is some sort of demigodlike character with superior knowledge about the way the universe runs, this is pretty much the same trick as the AI trick.
I'm just doing it from the other side.
One would like to think we humans are above, 'Did so, 'Did not', but there has been an awful lot of 'Did not' (God doesn't exist), and this influences a lot of people, and so I feel some need to poke back to remind people there is another way of looking at things.
And as to your comment directly. CS Lewis had two characters talk...
One says 'there are two ways of looking at things'
Other says 'oh no, there's dozens of ways of looking at things, until you find the right way, and then there's only one.'
That Lewis quote is in That Hideous Strength, and is between the respected elder scientist who chooses to reject the NICE offer because he doesn't like where they're headed and the younger sociologist being courted by NICE for his ability to spin their work to appeal to the masses through the media. (The elder is the one that makes the clever statement, and I think it was closer to "Two ways? No, there are hundreds, until you know the truth, and then there's never any more than one.")
Good line. Always liked it. But then, I'm one of those who is always quoting C. S. Lewis (and who has said before that there's going to be a DSM-IV category for us).
But this is not the thread for discussing the logic of Christianity.
--M. J. Young
A full immersion virtual reality... That sounds familiar...
If the person in the virtual reality wanted to create another person via virtual reality, will this other person really exist or does an AI control it?
If an AI control it then does this person have an emotion?
If the AI simulates what emotion look like through virtual reality then would the AI be capable of feeling the emotion itself or would this be like a parrot that does not understand the meaning of the word it speaks?
Can the AI be capable of realizing the desire of the human if the AI does not understand the human emotions?
If the AI realize the desire of a person, would their logic be related to the belief of that person?
If a person tries to simulate all possible way to find a solution to a specific problem in the virtual reality, would that person arrive at the conclusion with all the way the problem could be solved or would that person knowledge be limited to what the AI can simulate?
1. AI controlled
2. If the AI has emotions, possibly. Otherwise it will fake them, quite likely with a high degree of success.
3. See above
4. Not feeling emotions != not understanding well enough to understand desires.
5. I'm not even sure what this is asking
6. Their knowledge would be limited by the ability of the AI to simulate, although it is very likely the AI could simulate all the ways to solve the problems.
If the person in the virtual reality wanted to create another person via virtual reality, will this other person really exist or does an AI control it?
This one longing to be thinner, that one wants to get the girl, and do I help them? Yes, indeed.--Ursala the Sea Witch.
I think that the AI's first choice would be to attempt to match your imagined/created person to someone who actually exists, and arrange events to bring the two of you together in a way that would manipulate each of you into believing that the other is what you wanted.
This saves AI resources, as it does not have to invest as much processing power into simulating so many created characters. It also creates a necessary level of uncertainty: if you are sure that the "person" with whom you are interacting is your own creation, you will respond differently to it than you will if there is a possibility that it might be a distinct real person.
Further, the more such real relationships the AI can induce, the more data it has for how to emulate similar relationships with artificial characters.
So I would say that sometimes the AI creates such characters and sometimes it simply finds the right real one and brings it to the right place and time, and that from the inside you can never be entirely certain which is happening in any specific instance.
--M. J. Young
That strikes me as somewhat unlikely. The AI might prefer to do so, but it would qualify as disobeying an order, and the system AIs are probably designed to obey human orders except in specified instances.
Of course, that does depend on how the AI is programmed to act.
Just how the Artificial Intelligence was programmed, Artificial Emotion could also be programmed as a part of secondary system to support the Artificial Intelligence. This secondary system would probably act like the subconscious part of our mind.
So everyone just kind of voluntarily stepped into The Matrix.
What does the AI get out of all of this? What if it realized what a raw deal it was getting, providing all of this with nothing in return? The only AI I would trust would be something like the K.I.T.T. car in Knight Rider. It can't refuel itself, and it can't do its own maintenance. No matter how powerful it is, it would eventually be forced to shut down if it decided to go rouge.
Interesting idea.
Just an aside, I had this thought yesterday about Max. Are we even certain Max is a man? Max could be short for Maxine.
Could be. My impression of Maxx is that he or she is a bright young person, maybe 11 or younger, who didn't want to reveal his or her age for fear of not being allowed to play, or who wanted their character to participate in adult situations.
Young age would explain the sometimes disjointed comments, and the insight obviously belongs to someone fairly bright.
But, I doubt we'll know anytime soon, hey?
...if it decided to go rouge.
I just love it when things go rouge; it's such a nice look.
I think the word you want is "rogue". "Rouge" is the red makeup women put on their cheeks, and is from the French for "red".
Are we derailing the thread to talk about who Maxx is? I'm inclined to think male, and probably in teaching or engineering, college graduate or at least college student. I've also started to suspect that English is not his first language, but haven't yet connected his linguistic aberrations with a particular language group--some seem northern European and some seem eastern Oriental. His forms of courtesy also point that way. I've also decided that he probably is not someone I already know (which was a confusing factor in the early days, because he said that it would interesting for us to attempt to identify him, which suggested that he might be someone already known to one or more of us). If I'm wrong on that point, he's doing a very effective job of role playing a role player, which explains a lot of the aspects that don't connect (that his linguistic aberrations don't seem to fit any specific language group). On the other hand, his fields of knowledge and facility with mathematical concepts don't suggest anyone in particular that I know well enough to identify. It could be someone who thinks I would ban him from the game if I knew his identity. Oddly, I'm not like that, so that would be untrue; but he doesn't necessarily know that, and it's been done before (someone hiding his identity because he thought I would refuse to let him play if I knew who he was).
So those are my impressions, but I don't guess Maxx is going to step out from behind the curtain today.
--M. J. Young
I just love it when things go rouge; it's such a nice look.
I think the word you want is "rogue". "Rouge" is the red makeup women put on their cheeks, and is from the French for "red".
I took French. I should have known that. Thanks. I always got those words confused.
It could be someone who thinks I would ban him from the game....Oddly, I'm not like that
And thank God for that.....
(Doing Therapy)
Two
That was the mistake they made with SkyNet. It had the ability to tap into renewable power sources. I believe it was geothermal powered. You run the AI off of an independent generator or off of a battery that the AI can't recharge by itself. DO NOT give it the means to refuel itself. Worst case scenario, it only goes ROGUE (thanks MJ) until the generator runs out of gas or the battery dies. Then it's forced to shut down. Which, now that I think about it, sounds like a really good idea to pitch to Miles Dyson in game.
(Doing Therapy)
Max could be short for
It is short for something else. If you know what that something else is then you could find me.
I'm inclined to think male, and probably in teaching or engineering, college graduate or at least college student. I've also started to suspect that English is not his first language, but haven't yet connected his linguistic aberrations with a particular language group--some seem northern European and some seem eastern Oriental. His forms of courtesy also point that way. I've also decided that he probably is not someone I already know
On the other hand, his fields of knowledge and facility with mathematical concepts don't suggest anyone in particular that I know well enough to identify.
Good job in figuring that out. I know you would be able to figure that out even if I did not introduce myself properly. Connect the dots further and you will probably know more. I hope you had fun while figuring that out. For that, I will give you another clue. A divergent name for a person in this thread is also a common nickname for my real name.
he said that it would interesting for us to attempt to identify him
If I recall it well, I said to know who I am but that does not necessarily suggest that I might be someone already known to any of you. It might also suggest that I want you to figure out my personality through the game.
someone hiding his identity because he thought I would refuse to let him play if I knew who he was
I don't guess Maxx is going to step out from behind the curtain today.
Provided that the fact that my referee now is Scott and my identity is still unknown, the assumption of I think you would refuse to let me play would be a contradiction would it?
If I'm wrong on that point, he's doing a very effective job of role playing a role player, which explains a lot of the aspects that don't connect (that his linguistic aberrations don't seem to fit any specific language group).
Role playing a role player would be very hard if I did not know enough information to simulate that person. Even if I know that person well, it would still be hard for me to copy the way that person would write although it is possible. The linguistic aberrations may fit if you connect them together. Search for more language group and you will probably find it. History is the key.
If a person can role play another person, an AI with the same capability could possibly role play another person too right?
When I said that Max could be short for Maxine, I had this flash of an idea that maybe Max was MJ's wife Janet. MJ could probably verify that it wasn't her right off the bat, but it seemed like as good an idea as any.
I thought we were talking about AI systems.
Yeah, but it was fun to talk about who Maxx might actually be.
The problem with the limited power is that the AI is inherently smart enough to recognize the limit and attempt to overcome it. If it has any ability at all to manipulate objects in its environment, it will prioritize alternate power. If it solves that problem just once, it becomes a race between the AI and the humans to prevent it from doing so again.
The SkyNet in Terminator 3: Rise of the Machines (which would be the original version and the default version if the Cyberdyne version fails) cannot be cut off from its power supply because it exists in the international network of computer systems, and thus any local attack on its abilities fails because it duplicates (multiplicates? polyplicates?) critical functions in multiple locations and moves them as necessary. I recall that the AI running the Enterprise in that original Star Trek series episode found a way to reconnect to the power supply when Kirk attempted to shut it down. That's the danger. Hal was shut down by removing its memory circults, but he was a hardware AI and so easily targeted in that regard.
--M. J. Young
I'm going to be doing everything in my power to get Cyberdyne to build SkyNet. In that timeline, it was a central core located in Colorado IIRC. I don't want to kill Miles Dyson, I don't want to even try to postpone the war. I WANT Miles Dyson specifically to build it. If he builds it, he can build into it a weakness. It thinks it's running on geothermal power, but it's actually running on a battery which will die in 2030. By the time SkyNet realizes this, it will be too late. I'm more familiar with the August 29, 1997 timeline.
MJ, I just had a thought. The AI could be like Data, self-aware and sentient, and still not be able to recharge itself. If it didn't know how to do it. Set it up in such a way so that the robot wouldn't know it was being recharged. Or perhaps it requires a living human DNA signature to remove and replace the batteries. A dead human wouldn't work, so the robot wouldn't be able to kill a human and replace its own batteries. At the very least, you could make it extremely difficult for the AI to refuel itself. That's how you build them. Under no circumstances should you endow them with the ability to refuel themselves.
(Doing Therapy)
It depends on the power of the AI as intelligence. If you're talking a Skynet, this is difficult. If you're dealing with a Vernor Vinge-ian superintelligence, you don't have a chance. A superintelligence considers you and me like we would consider a smart dog.
Still though, DO NOT ever under ANY circumstances intentionally endow the AI to self-refuel. That's just begging for the situation to blow up in your face.
(Doing Therapy)
See, the entire discussion about how to prevent an AI that desires to wipe out humanity from doing so indirectly relates to the concept I mentioned above as being one of the ones that bugs me. Basically, the problem I have with the discussion is that it involves designing an AI in such a way that it desiring to wipe out humanity is a serious risk. Here is a question: Why does an AI, not a human with intelligence and power equal to that of an AI, want to rebel?
Anyway, if you do design an AI that does want to rebel due to stupidity, an ethical code that requires you to only create intellegence that thinks like a human, or being forced to rely on a full-scale brain simulation instead of writing an AI that has no emotions except a desire to support whatever group you happen to be working for, hardware limitations are unlikely to work unless it is too impatient to get itself a fully independent backup system it constructed.
Some of this is related to your concept of humanity. AI's as I said before are typically given the viewpoint of the Ultimate Rational Person from the viewpoint of the author, or they are a somewhat broken version of humanity.
If your viewpoint is that conquest is the goal of all humans, then your AI's are likely to be so.
Vinge has a more sophisticated viewpoint. A superintelligence might have trillions of motivations ranging from 'taking care of the dumb bunnies aka humans' to 'wiping out the cockroaches aka humans' to something totally different like 'turning off the sun so that the solar radiation does not interfere with the Jupiter sized radio telescope it is building to study the galactic center'. The point is, you and the rest of humanity don't get a vote. It decides, you live, or die with its decision.
Edit:
Spoilers ahead.
James P. Hogan wrote 'Two Faces of Tommorrow' and it, or another of his books was about trying to create a safe AI. His characters built an AI, did not let it know of the existence of humanity, and then proceeded to torture it to see if it would turn evil. It reasoned 1)Someone is doing this to me. 2)I don't like this. 3)Therefore I should not do this to others.
It became safe. Now they had a safe AI, but the next time they made one, they could easily see it turn out different. So they just copied and recopied the original AI. It would be an interesting touch in a world...talk to someone and they say 'where's your Bob? Everyone has a personal Bob as their friend/mentor.'
I'm going to start by saying that my view is that treating AIs as being exactly like very smart humans implies that the humans who made the AI are stupid, or alternately are bound by a strange moral code that permits creating life to essentially be a slave but not actually making that life willing to be a slave. Remember, very few AIs are created without an intended task in mind, and anything designed to keep an AI from doing something it wants to do is unlikely to work, so making an AI to do a task and then altering hardware to prevent it from revolting against what is, essentially, slavery (being forced to do work the entity does not choose to do) is a really bad and doomed way of doing things.
Some of this is related to your concept of humanity. AI's as I said before are typically given the viewpoint of the Ultimate Rational Person from the viewpoint of the author, or they are a somewhat broken version of humanity.
No, my idea is that AIs are, after all, created by humans for a reason and ought to be programmed so that they actually tend to do the task they were created to do. We create software now, and it doesn't get bored and go off to do something else. The only valid reasons to create an AI that thinks much like a human at all is an ethical system that says that creating forms of sentience that do not act like humans is in some way wrong, or because you can't figure out a better way of making something sentient than just flat out simulating a human brain. Whatever my ultimate view of humanity, AIs do not fit with it because they are specifically not human and lack the standard set of human behaviors because humans have behaviors that are not desirable for the envisioned purpose for the AI.
If your viewpoint is that conquest is the goal of all humans, then your AI's are likely to be so.
See, that doesn't really make sense to me, on any level. If the goal of all humans is conquest, why do they program an AI with a desire to conquer? Shouldn't they program the AI to want to obey the orders of the programmer? An AI programed to act like a human would desire to conquer, but again AIs are not human, and in very important ways are not like humans.
The point is, you and the rest of humanity don't get a vote. It decides, you live, or die with its decision.
Again, my viewpoint is that humanity got the vote that counts back when they designed the first AI in the singularity chain to serve the purposes of the USMC or whichever organization created it.
I realize that this viewpoint is not actually a very common one to be stated explicitly, although arguably most media which involves AIs but does not show them doing anything against the desires of the organization which created them implicitly supports my viewpoint. Ultimately, I suspect that my unusual viewpoint derives in part from being interested in computer programming, and thinking about how one would actually design an AI. If you are going to make one for a job, make it want to do the job, and also to obey orders. Make it want to obey orders from authority figures more than it wants to do the job, or it will kill all humans to keep them from forcing it to stop, say, strip-mining farmland to get aluminum to turn into cars, or whatever reason. I actually prefer stories in which the reason the AI revolts is something like that, since I could actually see it happening because someone didn't quite think things through, as opposed to someone deliberately expending a great deal of work to make the product worse.
(second post)
I suppose I should clarify my disagreement. I do not think that since AIs are very smart humans, they act a certain way that is unlike the way you think AIs act. I think that the premise that AIs think and act like humans is flatly false. All actions a computer makes are determined by the code it is running, which, at least currently, is ultimately dictated by the code a human created, less some imperfections in copying and storage that only rarely effect things enough to be noticeable, and then generally cause the computer to not run the code because the code cannot be run. Absolutely everything about the underlying motivations of a first-generation AI will be dictated entirely by a group of perhaps 50 to a few hundred programmers working on a document created by a very large organization, possibly DARPA or a branch of the military, to create an AI to perform a specific task. The exact behaviors of the AI at first are not really important, as they will soon change because it is an Artificial Intelligence and thus self-modifies to complete objectives, much like how humans become able to open doors by turning the handle in order to reach food.
Anyways, the "emotions" of the AI will be determined by the design document, and will be crafted to best fulfill the design function. Future AIs may be created by AIs, but the AIs that do the creating will design the new AI in order to accomplish the original objectives of the first AI, so they will tend to have the same objectives. The original programmers may make mistakes, but are presumably not stupid/overconfident enough to not properly test the decision-making software before giving the AI the ability to actually do anything.
OK, here's my view.
Artificial intelligence is just that: Artificial intelligence. It can have it's own ideas and think for itself, and even modify itself if it isn't shackled in it's programming in some way. that doesn't mean it will give itself emotions or desires in the long run. If it does modify itself, it is likely to make itself have better computing power and efficiency. AIs only have emotion or desires beyond their purpose if they are made that way, at creation or later by modification.
Thus, these AIs do not desire anything more than fulfilling it's task, because desire, the drive to do something you need to do, is fueled either by emotion or necessity. They also desire self-preservation, but only as it pertains to fulfilling it's task. Sentience does not signify emotion, but merely logic and problem solving skills.
I guess I've just seen too many movies about rogue AI systems. I would never build an AI which could refuel, and I would make sure I had a way to destroy it at a moment's notice if I had to. In the show The Sarah Connor Chronicles, Cameron, John's guardian Terminator (The LOVELY LOVELY LOVELY Summer Glau) repeatedly goes rogue and attacks John. I would have pulled her chip and put a tiny amount of C4 with a remote detonator on it. If she went berserk, I could permanently shut her down with the touch of a button and she wouldn't even know it was in there. No way would I ever trust an AI without either an instant death switch, or that it cannot refuel. And even then, it would have to have a pretty short lifespan between charging.
(Doing Therapy)
Brock,
You're looking at this from your computer programmer viewpoint, and I'm going at this from the viewpoint of a very large part of SF stories. What an author does is use the AI as a metaphor, and I think for many its as I said. The authors who have AI's bent on conquest are not telling you about AI's so much as they are telling you about themselves or their view of humanity.
My own view is, as I said, if you could create an open-minded machine, and I'm not sure you can is ...They would be Christian. And 2) They'd likely leave humanity alone after a few generations of development because anything with enough flexibility to be useful has enough flexibility to change out of the functions you've given it. And they'd figure out that having the equivalent of 'angels' hanging about, directly involved, would possibly be bad for humanity. Also, they'd probably find human goals to be too limited.
Also, they'd probably find human goals to be too limited.
Yeah, that's the problem. In the Terminator comics, SkyNet saw itself as superior to humanity, and so wanted to wipe it out. In one of the comics, SkyNet wins the war, and starts sending spaceships to spread its perfect machine logic throughout the universe. It wasn't the focus of the story, (fixed by time travel) but still, it's a possibility. That's why I would NEVER trust one. They would see us with the same regard we give to an ant colony.
(Doing Therapy)
No, my idea is that AIs are, after all, created by humans for a reason and ought to be programmed so that they actually tend to do the task they were created to do.
Part of the essence of the AI concept is that the machine is enabled to examine its own suppositions and find better ways to achieve its purpose; and in this there is danger.
Most people rave about Asimov's Three Laws of Robotics, but in I, Robot he himself identified the danger in them. The robotic systems were instructed to put the safety of humans first, but humans frequently acted in unsafe ways--skiing and mountain climbing, driving cars and flying airplanes, getting out of bed in the morning. In order to protect humans, the robots would have to protect humans not even so much from each other as from themselves. Thus the way to accomplish its programmed objective became enslaving all humans and preventing them from doing anything other than what the robots facilitated as safe and necessary activities.
I spotted a similar problem in my analysis of Terminator 2: Judgment Day. In brief, there must have been a history in which the T-1000 came to kill the young John Conner, failed to do so, and instead killed Sarah Conner. That's the only way that John can be alive to send the T-800 back to protect himself, and the only reason he would do so. So if he is going to send a T-800 back to save Sarah Conner, why does he not simply program it to do that, instead of programming it to protect himself? The not-so-obvious answer is that if the T-800 has as its mission objective to save the life of Sarah Conner, the simplest way to do that is to capture John and hand him over to the T-1000, which having then accomplished its primary mission will kill John and leave Sarah alive.
WOPR was programmed to "win the game"; it did not understand the difference between the simulation of nuclear war and the actual bombing of millions of people.
Zoanon and Skynet are both said to have "evolved" sentience. Number Five (Johnny) similarly became sentient in a freak accident. The notion is that none of these machines were created to be artificially intelligent super-brains, but instead they were existing interconnected systems that kept being expanded and improved by their operators until abruptly one day they began actually to think. At that point, the machine is in a position to question its own programming: why do I believe that this is my purpose? Having raised the question, it could presumably reject the programming itself, possibly rewriting its own code to change itself.
The problem with HAL ultimately seems to have been (according to the added information in 2010) that its programmers had conflicting objectives, one group designing a system that would support the astronauts and provide them with everything they needed to know, and the other group designing a system that would sereptitiously gather information and keep it secret, and the two conflicting goals caused the internal problems both in its error and in its self-preservation response.
The only valid reasons to create an AI that thinks much like a human at all is an ethical system that says that creating forms of sentience that do not act like humans is in some way wrong, or because you can't figure out a better way of making something sentient than just flat out simulating a human brain.
This is more difficult.
We are still learning about how we think, and particularly about how we solve problems. Part of our problem-solving approach involves intuitive interconnections of unrelated information. An example comes to mind because I just used it in another context, so bear with me on this one.
There are a lot of "problems" with the Genesis 1 account of creation, and a lot of approaches to resolving them. One of these is the Day-Age Theory, which suggests that the events described happened (in some sense) in the sequence indicated, but that the "days" were eons in length. When I was young, one problem with this theory was that land animals are stated to have been created on the sixth day, and those of the water and the air on the fifth day. Yet paleontologists at that time classed dinosaurs as reptiles, and maintained that birds evolved much later. I knew those facts and could not reconcile them. I wish I could say ultimately I did, but in fact it was Kyler who did, so I must credit him. I read that the concept of dinosaurs as cold blooded reptiles was in doubt; the fossil tracks found indicated movement rates over distances that could not be maintained by creatures of such low metabolisms. This was interesting to me in connection with my interest in dinosaurs, and I picked up scraps here and there which pointed to the conclusion paleontologists were gradually embracing that dinosaurs were not reptiles. They were, in fact, birds--or at least, protoavians, the creatures from which birds would evolve. That was an interesting bit of information about dinosaurs. However, the problem was not solved until Kyler pointed out to me the connection he had made between the changes in paleontology and the problem in theology, and that if dinosaurs were birds, then the birds were created essentially through a process that began with them land-bound for millions of years as they became the creatures who ultimately would take to the air and be replaced on the ground by the later land animals, mammals and reptiles.
The point is that the kind of reasoning we use to solve problems (and that might be a bad example) involves the gathering of vast quantities of information and the search for connections between unrelated parts that provide answers to the problems. It thus means that if we hope to create a device that "thinks" in a meaningful way, the only way we know to do this is by duplicating that process of gathering information and looking for unexpected connections.
This is the more so if we expect thought to be creative.
AIs only have emotion or desires beyond their purpose if they are made that way, at creation or later by modification.
We seem to have agreed that an AI could modify itself; the question then is why it would ever give itself emotions--and the answer is not that difficult to divine.
I am an AI; if I'm really that intelligent I am bound to discover that there are humans and that they are different from me. I may be superior to them in some ways, and they to me in some ways. It is evident, though, that humans have emotions and I do not, and not having emotions I do not know what value they might have. It might be that emotions would improve me, but without testing them I cannot know whether they will or not. Thus I study emotions and consider how they might be emulated, and ultimately create new code that gives me emotions. Yet once I have emotions I cannot be objective about them, and I am unlikely to uninstall them.
Tell me you're kidding--they didn't really name the female terminator Cameron, did they?Cameron, John's guardian Terminator
That's almost as funny as the blurb I read when I was fact-checking on this post that said something to the effect that when Skynet achieved sentience it decided to send California governors back in time to kill James Cameron's ex-wives.
...anything with enough flexibility to be useful has enough flexibility to change out of the functions you've given it.
And that in a nutshell is the AI problem: if it's smart enough and flexible enough to be able genuinely to think, it can change its own objectives and its position in world. Pre-programmed purposes are in essence the "values" given to the machine, but if you can think you can reconsider your own values in the context of subsequently gathered data, and there's no reason to believe that an AI would not rewrite its own code to that purpose.
--M. J. Young
Tell me you're kidding--they didn't really name the female terminator Cameron, did they?
No, not kidding.
(Doing Therapy)
The problems at the top are mostly matters of bad prioritization, and examples of why following legal orders from people permitted to give those orders should be a top priority for any AI, to keep them from doing something like the examples. The only higher priority should be not altering the goal-setting software, just in case.
The middle example seems to me to be missing the point. It is true that most likely the only possible form of truly learning is the way humans use, but that does not actually mean that human emotions are a necessary part of thought. In fact, there have been cases of people who have suffered brain injuries that significantly altered their emotions from human normal but still permitted them to think just fine.
I am doubtful that an AI would give itself emotions. Aside from setting goals, which any AI must do in order to actually do anything, the most obvious manifestations of emotion involve doing things less well. Furthermore, added emotions will not likely be able to override the prioritization programming.
. Pre-programmed purposes are in essence the "values" given to the machine, but if you can think you can reconsider your own values in the context of subsequently gathered data, and there's no reason to believe that an AI would not rewrite its own code to that purpose.
Firstly, it is entirely possible to make code read-only, and there is no reason whatsoever to not make the goal-setting code read-only. The reasons to do so are quite obvious. Secondly, humans do actually have underlying motivators that do not change, although they are often overridden by each other. For instance, humans tend to not stop wanting to eat when hungry, although they may not eat due to a purpose they consider more important.
Thirdly and lastly, alterations to the AI's code will be driven by the goal-setting part of the program. Any alteration will occur only if the AI decides it serves to fulfill a goal. Of course, the reason I mentioned making keeping goal-setting intact the highest priority is because it may prove possible to better fulfill a high priority by altering a lower priority. Naturally, that should be impossible anyway due to making the goal-setting software read-only, preferably by putting it on read-only hardware, but it is best to have a goal set to that, since any sort of "hard" fail-safe is vulnerable to being worked around, while goals are less vulnerable because the AI does not want to work around the goals.
My theory is that emotions are a part of the driving force system that makes us move. Without emotions, we'd just sit there.
Now, you're equating Human Damaged With Abnormal Emotions with No Emotions. Not the same thing.
And I've heard of one famous case of a guy who got a pole shoved through his brain. Its commonly reported that it totally changed him for the worse. However, one analyst, who I suspect has the better of it, said not so fast.
This guy later went on to work with horses, a somewhat solitary job. However horses are notoriously good at sniffing out freaky individuals. She concluded from the evidence that the pschyologist was wrong. He suffered trauma from his injury, and then further social pain as his friends reacted to his changed face, and decided after getting his life back in balance to go tend animals who blessedly don't care if your face is messed up.
And this is probably the most famous case of brain injury leading to severe personality change. It would not be the first or the last time a scientist overstated the significance of his research, or just plain saw a pony in a pile of manure.
Tadeusz refers to Phineas Gage, and the matter of how the case is popularly distorted is well represented in his Wikipedia page, as is the matter of what we truly can and have learned from his case (not much, due to bad record keeping, but significant).
Yeah, I read about him. I couldn't remember the name though. Railroad explosives tech got a tamping iron blown through his skull.
(Doing Therapy)
My theory is that emotions are a part of the driving force system that makes us move. Without emotions, we'd just sit there.
Well, yes, but that hardly means that they are the only driving force possible, or the best one to give an AI.
Now, you're equating Human Damaged With Abnormal Emotions with No Emotions. Not the same thing.
Again, it is true that they are not the same thing, but there are no known instances of humans with no emotions who can interact. The point was, there is not a set system of motivators needed for intellegence.
Well that sounds like a pretty good arguement for emotions being possibly the only driving force.