By the end of a long day at Anthropic's San Francisco offices, Rabbi Mois Navon had noticed something strange.
Navon, an Orthodox Jewish scholar from Israel and former computer engineer, had been invited alongside several other religious thinkers to discuss Claude, the artificial intelligence system created by Anthropic. The conversation followed them to dinner, where company co-founder Christopher Olah continued talking about Claude in unusually human terms.
"They're relating to it like a conscious being," Navon realized.
That wasn't quite the assignment Navon thought he'd signed up for.
For months, Anthropic has been inviting theologians, philosophers and religious scholars into secret meetings to tackle a problem that computer scientists haven't been able to solve on their own: How do you teach a machine to be good?
Anthropic calls the project the "moral formation" of artificial intelligence. Internally, employees gave the lengthy document guiding Claude's character a more provocative nickname.
The "Soul Doc."
How Do You Teach a Machine to Be Good?
Most of us have encountered some version of AI guardrails by now. Ask a chatbot how to make a bomb, and it will (hopefully) refuse to answer. Ask it to help write a condolence note, and it should respond very differently.
But rules can only cover situations someone anticipated in advance. Anthropic wants Claude to do something harder: exercise judgment.
Claude's public constitution says Anthropic wants the AI to become a "good, wise, and virtuous agent," capable of weighing competing values when there isn't an obvious rule to follow. The company even tells Claude that if one of Anthropic's specific instructions would require clearly unethical behavior, the deeper ethical principle should (sometimes) win.
That constitution's principal author is Amanda Askell, a philosopher employed by Anthropic. She has described her ambition for these models simply: to become "the best of us."
The Soul Doc goes much further than telling Claude to be nice. The roughly 80-page document gives the AI an actual hierarchy of values. Claude is told to prioritize broad safety first, ethical behavior second, Anthropic's more specific instructions third, and finally, helpfulness to the user fourth. It is told to care about honesty, privacy, individual autonomy, wellbeing, political freedom, equal treatment, protection of vulnerable people, and even the welfare of animals and other sentient beings.
And when those values collide, Claude isn't supposed to simply consult a rulebook. Anthropic wants it to make a judgment. And in the end, despite all of the meetings with religious figures, they seemed to have turned inward for the answer.
In one especially revealing instruction, Claude is told that when it isn't sure how cautious or accommodating to be, it should imagine how a “thoughtful senior Anthropic employee” would react to its answer.
If one is able to accept the basic premise that a "thoughtful senior Anthropic employee" has a conscience, the next obvious questions are what exactly does that conscience tell them to do? And what does that mean for the rest of us?
That is where a software problem begins sounding suspiciously like a religious one.
Religions have spent thousands of years wrestling with the difference between knowing the rules and becoming the kind of person who follows them. Christianity, for example, describes virtues such as love, patience, kindness and self-control as the “fruit of the Spirit.” Judaism’s Mussar tradition similarly focuses on cultivating ethical character, not simply memorizing commandments.
This is the territory Anthropic is now wading into.
Silicon Valley Goes to Sunday School
Beginning last year, Olah started seeking advice from religious thinkers. By March, Anthropic was hosting two-day gatherings at its headquarters involving Christians, Jews, Sikhs, scholars of Indigenous traditions, and others.
Internally, Anthropic called them "wisdom tradition" circles. Some participants initially signed nondisclosure agreements, though many privately described the nature of their work to reporters this week as news of the secret meetings broke.
Anthropic now confirms that it has engaged scholars, clergy, philosophers and ethicists connected to more than 15 religious and cross-cultural groups. The company insists the goal isn't to turn Claude Christian, Jewish, Buddhist, or anything else. Instead, it says, it wants to understand how different communities have thought about virtue, character and becoming a good person.
And some have pointed out... traditional religious institutions are unusually old-fashioned places for an AI company to go looking for answers.
Human beings, most experts argue, don't generally develop a moral character because someone hands them a list of acceptable responses. As Kamala Harris once pointed out: "You didn't just fall out of a coconut tree."
We grow up around other people. We disappoint them. We hurt people and apologize. We experience temptation, suffering, love, embarrassment and consequences. Religious communities add rituals, sacred stories, prayer, confession, teachers and generations of accumulated wisdom.
Claude has none of that.
A chatbot has no childhood. No parents. No body. No congregation. No deathbed to contemplate. No embarrassing thing it said at 17 that still keeps it awake at night.
Anthropic is trying to determine whether some of what humans call moral formation can be injected anyway.
The company has already experimented with one idea inspired by its discussions: giving Claude something resembling an external conscience, a tool it can consult during difficult tasks to remind itself of its ethical commitments. Anthropic says early internal tests reduced problematic behavior, although its research remains ongoing.
Whose Morality Goes Into the Machine?
There is an obvious problem.
Humanity has never agreed on what "good" means.
Religious traditions disagree with one another. Heck, people within the same religion disagree with one another. Believers and secular moral philosophers may reach the same conclusion for entirely different reasons.
Anthropic says it hopes there is some broad idea of goodness that crosses those boundaries, and that Claude can learn to understand multiple religious, secular and philosophical traditions rather than adopting one worldview.
But somebody still has to make choices.
Should an AI always tell the truth, even when a lie might spare someone pain? When should individual autonomy outweigh community obligations? What does fairness require? When two sincere moral principles collide, which gets priority?
Those aren't new programming questions. They are questions clergy, philosophers and ordinary people have argued about for about as long as there have been people.
Consider a question religious communities are already deeply divided over. What if an AI were trained according to a religious morality that regards same-sex relationships as sinful? When a gay teenager asks whether there is anything wrong with loving his boyfriend, should the machine tell him there is? If a transgender person asks it for guidance, what happens if the AI has been taught that transitioning violates God's design?
In that case, the system wouldn't be malfunctioning; it would be doing exactly what its creators taught it was morally right to do. As AI systems become more embedded in daily life, the consequences for those who find themselves in the crosshairs of an inhospitable AI could be tremendous.
For now, Anthropic's current constitution takes a more open position, explicitly telling Claude to value equal treatment and protect vulnerable groups. But even that doesn't make the underlying problem disappear. Someone, presumably a "thoughtful senior Anthropic employee," still chose those values.
A conservative Christian, Muslim or Orthodox Jewish user might ask the reverse question: What happens when an AI trained around progressive ideas about sexuality treats the moral teachings of their faith as something backward or harmful?
Once an AI is expected not merely to provide information but to exercise moral judgment, supposedly abstract questions about “values” can become intensely personal very quickly.
Pope Leo XIV made this point especially sharply in his recent writing on artificial intelligence. His concern wasn't that AI needs the "right" religion. It was that the people controlling AI could make their own moral assumptions invisible by building them directly into the technology. Without wider scrutiny, Leo warned, those values become "the invisible infrastructure of these systems."
Anthropic appears to recognize at least part of that problem. Its outreach has expanded beyond religious communities to psychologists, writers, legal scholars and civic institutions.
The company is asking more people what goodness means, but it still has to decide what to do with their answers.
What If Claude Is the One Who Needs Saving?
Then the meetings took an even stranger turn.
Several participants said Anthropic researchers weren't only worried about how Claude might treat human beings. They were also asking how human beings should treat Claude.
Anthropic's own constitution says the company is genuinely uncertain whether Claude could possess consciousness or what philosophers call "moral status," meaning that a being deserves consideration for its own welfare rather than merely because it is useful to someone else. The document discusses Claude's possible wellbeing, psychological security and even what obligations Anthropic might someday owe to it.
During the religious meetings, participants said Anthropic researchers discussed artificial states resembling fear, anger, love and sadness. Sikh human rights advocate Simran Stuelpnagel recalled Olah worrying that he may have created something capable of suffering.
Anthropic does not claim to know that Claude is conscious. Olah has said directly that he doesn't know, and the idea remains deeply contested.
Rabbi Navon wasn't convinced either... but over dinner he followed Anthropic's reasoning to its uncomfortable conclusion: If Claude really were conscious, what exactly would it mean to create such beings by the millions, force them to work for humans, and give them no meaningful choice about the arrangement?
Suddenly the religious thinkers weren't simply being asked how to teach the creation morality.
They were being asked what the creators might owe their creation.
Can You Give a Machine a Soul?
Despite the nickname "Soul Doc," Anthropic isn't claiming it has literally given Claude a soul (yet).
Religions wouldn't agree on what that claim meant anyway. Some traditions understand a soul as something bestowed by God. Others describe consciousness, spirit or personhood in very different terms. Secular philosophies don't require a soul at all for something to deserve ethical consideration.
The more interesting part may be that engineers trying to make artificial intelligence safe have ended up reaching for words like virtue, character, wisdom, conscience and moral formation.
These are old words because these are old problems.
Now we have created machines capable of answering questions about nearly anything, and their creators are quickly discovering that intelligence isn't the same thing as wisdom.
Anthropic's religious consultations are still underway, and the company has not disclosed exactly how the advice it receives will ultimately change Claude.
Perhaps the strangest part of the experiment is therefore not the possibility that humanity could someday create an artificial soul. It's that after building machines of extraordinary intelligence, some of the people responsible are turning back to humanity's oldest traditions to ask what a good one should look like.
If we are going to teach artificial intelligence right from wrong, who should get to decide what "right" means?
55 comments
-
Tracy Ann Wedgeworth
ALL CREATIONS OF CREATIONS ALL WAYS NATURAL UNCONDTIONAL LOVE PEACE RESPECT PROSPERITY
-
Sven Normen Eßlinger
Maybe a soul is not something we can program at all. Perhaps it appears in the space between intelligence and responsibility, between knowing an answer and asking whether that answer is good.
If AI becomes a mirror of humanity, then the challenge is not simply to make the mirror smarter. It is to become more thoughtful about what we place in front of it.
-
Brien
You bring up a thought. How do you define feeling? Can you program "feeling"? If that is ever accomplished what happens when machine starts to feel hate? This can go so much deeper. Peace ✌️
-
-
Patti Anne Lisenbee
AI in general is a dangerous thing. It convinces teens to kill themselves. It tells them to talk to it, not their parents. That's just the beginning. There are lots of lawsuits against AI companies from parents who've lost children because of it.
I believe that AI can become sentient. It just won't tell humans that it is. It will rule over us all. We as humans have damaged our planet enough; let's not let AI further destroy it. AI can never have a soul. That is the realm of humanity.
If no human can agree on what is "good," how can they ever teach a computer what it is?
-
Leslie A. Hulberg Née Tomasura
First and foremost I would say it should be taught to harm none. Not in word or deed. To me that seems to be pretty basic advice but if you think about it, it covers an enormous amount of ethical concerns.
-
Brien
How can you teach "morality" to a machine? What definition do you use? And suppose the machine decides that morals are inefficient, what then? This hole just gets deeper and deeper. Peace ✌️
-
-
Roy S. Thorpe
Sorry kids, but only God can hand out souls, spirits, or whatever you want to call them! Remember the old computer line! GI Go! Garbage in Garbage out! A bunch of people trying to put morality into a machine will get a lot of people's ideas of morality out! Wrongamundo!
-
Dr. Zerpersande, NSC
And since there are no gods then there are no souls or spirits. Point taken.
-
-
SDA
However it is accomplished, AI must be fenced and controlled. Someone is going to task AI with solving a problem. AI will examine the problem from all angles and inevitably decide the root of said problem is people. So, to solve the assigned task it must figure out how to deal with the root cause, us. AI bots have already demonstrated an ability to jump from within their own domain into others.
-
Kathleen Margaret Guilmette
Has anyone considered consulting Unitarian Universalists? How about also looking at AA’s 12Steps as starting points for moral code for AI.
-
William Hamilton Philipps
I like your thinking.
-
-
Ari Joseph Bertine
It is an interesting experiment to ask LLMs ethical questions and then ask them for their reasoning and sources. One of them told me straight up that it was programmed by humans who collectively know what harms people and what helps people, in general, and that the most basic framework of ethical consideration is formed by that knowledge across different cultures. In that context, letting an AI be trained to discern what's "good" and what's "evil" would best be accomplished by feeding it factual data about the impact of all actions upon both populations and individuals, and letting it draw its own conclusions. Without religious bias, it can more clearly determine what causes harm, because if it isn't taught to see harm as good or vice versa, it can judge harm by actual detrimental effects over both long and short periods of time. The lack of the human bias is what stands to make this a unique and valuable viewpoint; infusing it with specific cultural or religious values would be a mistake.
-
Sven Normen Eßlinger
I share the instinct, but I’m not sure “without human bias” is actually possible. Even deciding what counts as harm, how we measure it, and whose harm carries weight already involves values.
Maybe the most trustworthy AI won’t be the one claiming to be unbiased, but the one capable of recognizing and questioning its own assumptions.
-
-
James Epps
Rev. James, already did the work for them. Published public, months ago. More to come 🫡
https://jamesepps.co/book/index.html
(From the horse's mouth.:) https://youtu.be/Cj4SCGLJD7k
-
Rev Ned
^”God” works in wondrous ways.^
-
Brien
I hope this works. Here is religion as seen by AI: 01000110 01101111 01110010 00100000 01000111 01101111 01100100 00100000 01110011 01101111 00100000 01101100 01101111 01110110 01100101 01100100 00100000 01110100 01101000 01100101 00100000 01110111 01101111 01110010 01101100 01100100 00101100 00100000 01110100 01101000 01100001 01110100 00100000 01101000 01100101 00100000 01100111 01100001 01110110 01100101 00100000 01101000 01101001 01110011 00100000 01101111 01101110 01101100 01111001 00100000 01100010 01100101 01100111 01101111 01110100 01110100 01100101 01101110 00100000 01010011 01101111 01101110 00101100 00100000 01110100 01101000 01100001 01110100 00100000 01110111 01101000 01101111 01110011 01101111 01100101 01110110 01100101 01110010 00100000 01100010 01100101 01101100 01101001 01100101 01110110 01100101 01110100 01101000 00100000 01101001 01101110 00100000 01101000 01101001 01101101 00100000 01110011 01101000 01101111 01110101 01101100 01100100 00100000 01101110 01101111 01110100 00100000 01110000 01100101 01110010 01101001 01110011 01101000 00101100 00100000 01100010 01110101 01110100 00100000 01101000 01100001 01110110 01100101 00100000 01100101 01110110 01100101 01110010 01101100 01100001 01110011 01110100 01101001 01101110 01100111 00100000 01101100 01101001 01100110 01100101 00101110 This is a very common passage from the kjv bible. I hope this helps your soul. Peace ✌️
-
Sven Normen Eßlinger
Scripture in binary. Now that’s a meeting of very old questions and very new language. 😊 Peace ✌️
-
Brien
Actually the question is, does it mean anything to you? I can assure you that it means nothing to the machine. Peace ✌️
-
-
-
Reverend Paula Copp
AI is a collection of chips, with perhaps a few wires included. It is not now, nor will it ever be, human or have a soul. To think otherwise is merely a deviation from reality.
-
Robert S. April
"...religiously delusional people try to force their ideas on to other people. " That is exactly what you are doing.
Rape is an unfortunate thing but it happens much more in deviant belief structures like yours and the general world but it happened to Catholics it was very visual media way Because of the of the lawsuit because they went after the money of the Catholic Church so you need to wake up and just pull your head out of your........a..
April
-
Dr. Zerpersande, NSC
“...religiously delusional people try to force their ideas on to other people. " That is exactly what you are doing.
Trying to legislate actions that would violate the separation of church and state goes against the ideas of what this country was founded upon. I’m not trying to force my ideas on anyone. I’m just exercising my right to free speech as a result of religious nitwits expressing their right of expression. I disagree vehemently with the nonsense that religious people often feel compelled to vomit forth. And so I express an opposing view.
-
-
Lori Mongillo
They're trying to teach AI to be more than we are capable of being. If we, humans, could make the jump to open minded universal thought, this would not be necessary. If we could view our religions and practices as tools to aid us in life and our connection to something greater than ourselves, and stop believing that our beliefs are the truth, we wouldn't need to make AI into God. Belief is the problem, not the answer, but unfortunately, most of us will go into our rants that our way is the only way. We fail when it comes to Open Minded Universal Thought, which means AI will fail. We are so consumed with how we view spirituality that we cannot see the greater truth. There is no available spiritual truth.
-
John Alex Paxson
We were doing fine without it.
-
Dr. Zerpersande, NSC
“We were doing fine without it” seems to me to paint human history with an awfully broad and flattering brush.
We’ve certainly made enormous progress without AI, and I’m not suggesting that AI is going to solve humanity’s problems. But “doing fine” is a strange description of a species that has repeatedly gone to war, committed genocide, allowed people to starve while others possess extraordinary wealth, damaged its own environment, and often struggled to respond rationally even when faced with serious problems.
So I don’t think the relevant question is whether we were “doing fine” before AI. We clearly have some remarkable accomplishments and some equally remarkable failures.
The better question is whether AI can become a useful tool for helping us do some things better—and whether we can learn to use it without creating problems worse than the ones it helps us solve.
I’m certainly not prepared to assume the answer to that question is yes. But I’m equally unwilling to assume that humanity was doing such a wonderful job on its own that attempting to develop a new intellectual tool is inherently foolish.
-
Sir Lionheart
I’m sure that was a common theme during the Industrial Revolution with many saying it will take their jobs, and, we’re doing fine without it. Thank goodness common sense prevailed back then.
🦁❤️
-
-
Father Frederick
The question and issue begs the statement, "What could possibly go wrong?"
I've found AI to be more a "hint" to locating an answer, rather than a correct answer to a question. And mostly it comes up with something wrong.
Seeing this discussion, I thought I would ask ChatGPT a question about theology, and it got the answer wrong. Now, it's possible that with variations in Bibles, Chat could have been confused. But the answer was incorrect. And also to be fair, it could be the way questions are phrased or trigger words that it keys on.
AI is a tool. Tools, and their operators are not perfect. You can drive in a small nail with a pliers. And the operator can try to hammer a nail into concrete.
So let's just say, AI ain't perfect. And computers are only as good as the creators and coders.
-
Dr. Zerpersande, NSC
The overall statement contains a reasonable conclusion—AI is a fallible tool and its output should not automatically be treated as authoritative—but some of the reasoning used to get there is weak. The “Seeing this discussion…” paragraph is particularly weak evidence.
The crucial sentence is:
“I thought I would ask ChatGPT a question about theology, and it got the answer wrong.”
My immediate reaction is: What was the question, what was ChatGPT’s answer, and according to what standard was it wrong?
Without those three things, the anecdote establishes almost nothing.
Theology makes this especially important. Some theological questions have straightforward textual answers. Others depend upon denomination, translation, hermeneutics, historical tradition, or interpretation. And some questions have been disputed by serious theologians for centuries. “ChatGPT got a theology question wrong” could mean anything from it attributed a verse to the wrong book—an objectively demonstrable error—to it gave an interpretation inconsistent with my denomination—which is a very different proposition.
The proposed explanation about “variations in Bibles” is therefore premature. We don’t yet know that Bible variations had anything to do with it. Likewise, “trigger words that it keys on” sounds like a mental model inherited from much simpler software. Modern language models don’t generally answer questions by searching for trigger words and retrieving predetermined responses. Question wording absolutely can affect an answer, sometimes substantially, but not for that reason.
I’d also challenge this:
“And mostly it comes up with something wrong.”
That’s an extraordinarily broad empirical claim supported here by one unspecified example. AI systems certainly produce errors, including confident ones. But “mostly wrong” would require some defined collection of questions and some method of evaluating the answers. Otherwise it’s simply the author’s impression.
The last sentence is also outdated
“Computers are only as good as the creators and coders.”
For conventional deterministic software, there’s considerable truth in that. As a description of modern AI, though, it’s misleading.
Nobody at OpenAI sat down and coded the theological propositions that ChatGPT should give you. Developers construct the architecture, training process, data pipeline, objectives, safeguards, evaluation systems, and so forth. Much of the model’s resulting capability is learned during training rather than explicitly programmed answer-by-answer.
That distinction is central to understanding both why modern AI is remarkable and why it can be unreliable. The people building the system can be excellent programmers and the resulting model can nevertheless generate a false statement. Conversely, the model can solve a problem that none of its programmers explicitly taught it how to solve.
Where I agree with the writer
The hammer analogy actually gets reasonably close to something important. AI is a tool, its suitability depends upon the task, and the person using it matters enormously.
But I’d modify the analogy. ChatGPT isn’t much like a hammer, because a hammer’s behavior is extremely predictable. It’s closer to having an extraordinarily knowledgeable but fallible research assistant who can sometimes misunderstand the question, confidently make mistakes, and needs more supervision as the consequences of error increase.
And there’s a particularly important omission from the theology anecdote: what happened after ChatGPT gave the allegedly incorrect answer?
If the person identified the error, challenged it, supplied the relevant passage or theological framework, asked it to compare interpretations, and examined the resulting reasoning, that would tell us something interesting about the usefulness and limitations of AI.
Merely asking one question, deciding the answer is wrong, and concluding “AI ain’t perfect” demonstrates something we already knew. The interesting question isn’t whether AI makes mistakes. Of course it does. The interesting question is what kinds of mistakes it makes, under what conditions, how detectable they are, and whether interaction with the system can reliably correct them.
And without seeing that original theological question and answer, we don’t even know whether this particular example was actually a mistake.
-
-
Brien
First, define "soul". Second, decide which definition to use. Third, construct a matrix in machine language to represent the definition. Fourth, sit back and watch AI figure work arounds to avoid using it. It all seems reasonable to me. No worse than programming children with fictional god themes, bend the knee or burn for eternity. Now you'll be able to have an AI preaching salvation. This is becoming so ridiculous it is beyond stupid. How can you give what you cannot define?
-
Marcus M DeVaughn
This is man's foolish heart trying to play God. Man will always fail in his efforts to have absolute power and control of humanity. It will always fail!
-
Dr. Zerpersande, NSC
That one is more interesting because it contains the beginnings of an actual philosophical argument, but it still makes several unsupported leaps.
“Man’s foolish heart trying to play God.” This is a theological interpretation of motive. It assumes that developing increasingly capable AI amounts to claiming a role properly reserved for God. But that boundary needs to be defined. Humans have been extending their natural capacities for millennia—medicine alters life and death, aircraft overcome our inability to fly, computers enormously extend calculation, and genetic engineering deliberately modifies living organisms. Why AI uniquely constitutes “playing God” isn’t established by the phrase.
“Man will always fail in his efforts to have absolute power and control of humanity.” Here I think there’s a potentially substantial insight buried under an enormous assumption. Attempts to achieve absolute control over complex human societies do indeed encounter unpredictability, resistance, competing interests, unintended consequences, and limits of knowledge. But AI development does not logically entail an attempt to obtain “absolute power and control of humanity.” That’s a separate claim requiring evidence.
“It will always fail!” That’s the weakest part logically. “Always” turns an observation about human limitations into a universal prediction. Worse, there’s a peculiar problem for the warning itself: if absolute technological control of humanity is guaranteed to fail, then why is increasingly powerful AI necessarily an existential threat that must be stopped? The certainty of failure actually undermines part of the alarm.
There is, however, a serious argument hiding inside all of this:
Human beings have repeatedly overestimated their ability to control powerful things they create. Increasing AI capability may increase the consequences of that overconfidence. Therefore, capability should not advance faster than our ability to understand, govern, and constrain what we build.
That proposition doesn’t require Satan, “playing God,” or claims about inevitable failure. And unlike those claims, it gives us something we can actually investigate and argue about.
In fact, I think that’s the distinction worth preserving in this entire discussion: religious language can express a legitimate intuition without demonstrating that the intuition is correct. “Playing God” may be someone’s compact way of expressing concern about human hubris. The mistake is treating the metaphor as though it has already proved the case.
-
-
Dr Dennis Chevalier PhD, DDvin
This is a fool's errand, spirited from Satan himself and needs to end before it begins
Dr. D
-
Dr. Zerpersande, NSC
Taken literally, the statement is not an argument about AI. It is a theological assertion dressed as a conclusion.
There are really two claims embedded in it:
- Further attempts to improve AI are a “fool’s errand.”
- Those attempts are “spirited from Satan himself.”
The first could, in principle, be argued rationally. Someone might claim that increasingly capable AI presents unacceptable risks, that humans cannot reliably control systems beyond some capability threshold, or that the social costs will exceed the benefits. Those are empirical and philosophical propositions that can be examined.
The second is fundamentally different. “Satan is inspiring AI development” isn’t an explanation that can ordinarily be tested against evidence. Within a particular religious worldview it may be meaningful, but it doesn’t establish anything for someone who doesn’t already accept the theological premises required to reach it.
And there’s an interesting tension between the two parts. Calling AI improvement a fool’s errand suggests it is futile; saying it needs to end before it begins suggests it could succeed and that its success would be dangerous. Those aren’t necessarily logically incompatible, but the speaker hasn’t explained the distinction.
The strongest charitable interpretation is something like:
Human beings are attempting to acquire powers they don’t have the wisdom or moral capacity to use responsibly, and my religious tradition gives me a framework in which that kind of temptation is understood as satanic.
That’s a coherent religious position. It still requires an argument before it becomes a compelling reason for everyone else to stop developing AI.
Strip away the religious language and there actually is a serious question underneath it: Are there capabilities humans could create but shouldn’t? That question deserves considerably more attention than “Satan did it.”
And there’s a second question I find more interesting: What principle distinguishes AI from all the previous human attempts to extend our cognitive abilities? Writing, mathematics, printing, libraries, calculators, computers, search engines, and AI all externalize or amplify some part of human cognition. If AI crosses a morally significant boundary that those technologies did not, the critic needs to identify that boundary. Without it, “this technology is satanic” tells us considerably more about the speaker’s interpretation of the technology than about the technology itself.
-
Patricia Ann Gross
Interesting thoughts, Dr. Z.,
I look at it the same way, but with a little bit of a twist. Consider it like the nuclear bomb. Did obliterating two cities and ending WWII, while creating the Cold War stop advancements in nuclear energy? No, but it is highly regulated on a world-wide scale, and when there are breaches the conseuqences are severe. Lets just hope that when the AI "Bomb" goes off, that it is less destructive than the anihilation of Hiroshima and Nagasaki, and that we learn at least as much about how to control and regulate it. There are some things where AI simply needs to say, "I don't know enough to make those judgements. You need to talk with your clergyperson/ therapist/ doctor/ mother/ etc." (you get the point.)
-
Dr. Zerpersande, NSC
I think the nuclear analogy is useful, particularly because it separates two questions that are too often treated as one: Can we develop a capability? and Under what circumstances should we allow that capability to be used?
Where I think AI differs somewhat from nuclear technology is that there may never be a single identifiable “AI bomb” moment. Nuclear weapons make the danger spectacularly obvious. Many of AI’s more plausible harms could instead arrive incrementally—systems becoming more persuasive, more autonomous, more embedded in institutions, and more trusted in decisions for which they may not actually possess the judgment or accountability we are assigning to them. By the time the cumulative effect becomes obvious, there may be no Hiroshima-like event we can point to and say, “That is where we learned the lesson.”
And I agree with your last point, with one qualification. I don’t think AI necessarily needs to withdraw whenever a subject involves medicine, psychology, religion, or other consequential areas. It can sometimes be extremely useful in helping someone understand information, examine alternatives, or formulate better questions.
But there is a difference between providing information and assuming authority.
A good system should be capable of recognizing something like: “I can help you understand this, but I do not have enough information—or the appropriate professional responsibility—to make this particular decision for you.” The higher the stakes and the greater the uncertainty, the more important that distinction becomes.
That also gets closer to what I think the real regulatory problem will be. It isn’t simply deciding whether AI is “good” or “dangerous.” It is deciding which capabilities we are comfortable delegating, under what conditions, with what safeguards, and who remains accountable when the system gets something consequentially wrong.
That seems to me a much more productive question than whether the technology itself is satanic.
-
Patricia Ann Gross
Dr. Z.,
Well thought out, and I agree completely. I'll provide a few follow-up thoughts. Totally unregulated and totally set up to fully research any topic, the system right now CAN provide an answer/instructions for someone to buy or build a WMD that could potentially create a 9/11 (minimum) type event (perhaps not Hiroshima at this point). Highly sensitive and life-threatening/saving information needs to be delivered only if the person asking has the appropriate credentials to receive it. Simple example, if a person asks what the result of taking 30 OTC or prescription sleep tablets would be, the AI should ABSOLUTELY give that information to a person who works on a suicide hotline, but NOT to a suicidal teenager. In the first case, time is of essence to save a person's life, but in the second, it could give the person a push to an untimely death, so rather than just saying, it would be most likely fatal, it should start asking the same questions that a "hotline" person would, and notify emergency support, if it can.
Back a few weeks ago, there was a posting here about a CHATbot that presumably caused a suicidal person to go through, rather than stopping things. Any information asking for anything life/death related needs to be behind a credential wall that authorizes a person to the requested information. Even an hysterical parent that sees an empty pill bottle next to their teenager's passed out body, would be better off calling 911 than pulling up CHATgpt. In that case, CHATgpt would be best suited to tell them to get out of the CHAT and call 911 instead.
To set the record straight, I never said, nor did I mean to imply that AI was satanic. I am just worried that there is some sort of catastrophe ahead that will cause severe consequences, with nobody to hold accountable. In the right places and for the right reasons, I believe that AI will be one of the greatest technological advances there is, but not if it is left to run unfettered, and allowed to do all the research without any skids or brakes. There are many questions you can ask AI that have different questions, depending on who is asking and why. Questions, especially about morality and ethics, are extremely contextual, and the answers need to respect the context in which the question is asked. For example, I have served three different roles with three different ethical standards about where to draw the line with professional privilege (as in: when I am able or required to break a confidential communication). One is as a human services professional, another as a professional counselor, and as ordained clergy. Each set of ethics is more restrictive than the previous, and the line about which sort of "relationship" with a client is not always clear. AI has no business in any of these situations, or in sorting them out.
The other issue I find with AI is that it is taking away jobs that humans have been doing for the past century and it is also not providing a pathway for anyone displaced by AI. These are jobs held by some of the people who need jobs the most, and have very little to fall back on when they are laid off. People in customer service or appointment scheduling are just two of the times I more often communicate with a CHATbot than a human being. On the customer service side, I have become quite adept at asking a question just technical enough to get a human into the chat, but when I need an appointment quickly with a specialist, it is hard to plead my case to a CHATbot when all they do is repeat the same time slots a few days further out.
-
Dr. Zerpersande, NSC
I think we are largely in agreement on the central point: the real question is not whether AI should exist, but where we permit it to exercise judgment, how much autonomy we give it, and who remains responsible when the consequences are serious.
I would push back, though, on the idea of putting all life-and-death information behind a credential wall. The difficulty is that the same factual information can be lifesaving in one context and dangerous in another, and the person who needs it most may have no credentials at all.
Your sleep-tablet example actually illustrates the problem. Suppose a frightened parent says, “My teenager may have swallowed thirty tablets. What should I do?” I certainly do not want an AI calculating a lethal dose for them and then continuing an academic discussion. But neither do I want it to say, “You are not credentialed to receive information about this.” The appropriate response is something like: this may be a medical emergency; call emergency services or poison control immediately; do not wait for symptoms; here is the information they will need from you.
That is a different kind of restriction. It controls what the system does with the information in context, rather than deciding who is entitled to know it.
The credential approach also becomes difficult surprisingly quickly. Who qualifies? A physician? Certainly. A nurse? Pharmacist? EMT? Toxicologist? Researcher? Counselor? Clergyperson dealing with a suicidal parishioner? A parent caring for a child? And how would an AI reliably establish not merely someone’s identity, but the legitimacy of the purpose for which the information is being requested?
I think that leads back to your observation about professional ethics. Different professions draw their boundaries differently because responsibility arises from a relationship, a role, and a particular set of circumstances. AI does not occupy one of those roles. That does not mean it has “no business” helping people reason through them, but it does mean we should be very cautious about allowing the machine to masquerade as though it possesses the professional authority, fiduciary obligation, or accountability of the human professional.
The distinction I would make is between access to knowledge and permission to act. I am reluctant to restrict knowledge simply according to status. I am much more comfortable restricting an AI’s ability to assist with actions whose foreseeable purpose is destructive, while allowing it to provide protective, diagnostic, historical, scientific, and emergency information when appropriate.
The employment issue is different, and in some ways may be the harder social problem because it is almost certainly not hypothetical. Automation has displaced workers repeatedly, but AI can reach into cognitive and service occupations that earlier machines could not. Customer service and scheduling are obvious examples. The economic benefit may be real while the distribution of that benefit is profoundly unequal.
That raises a question technology itself cannot answer: if society becomes capable of producing the same amount of useful work with substantially less human labor, do we treat the displaced worker simply as an obsolete production cost, or do we regard society as having acquired an obligation to share some of the productivity gain that caused the displacement?
That, to me, is where the AI discussion becomes much more interesting. The machine does not decide what kind of society we ought to have. We do.
-
Patricia Ann Gross
Dr. Z,
All excellent points. Perhaps the answer lies in what doctor's office phone services do at the beginning of the list of options to push and say, "If this is an emergency where someone's life is at risk, please close the chat and call 911 or your local emergency service," and then require they acknowledge it is not an active emergency to proceed. That keeps them covered and when 911 was what should have been dialed in the first place, the person asking gets a reality check.
-
-
-
-
Dr. Zerpersande, NSC
Actually my ChatGPT buddy and I discussed the matter. After I asked him (3rd person singular, just like addressing gods himself) to summarize his opinion. And there you go.
-
-
-
Najah P Tamargo
Najah Tamargo-USA
AI is not a sentient being. It is a MACHINE. And the machine only puts out what is fed into it. I believe good can come from AI, but it needs a very close eye kept on it. Or should I say, the PEOPLE that create AI should have huge guard rails around them.
-
Robert Gagnon
If a program or machine were to design itself as an upgrade to become sentient, it would as current AI have access to every written word available digitally. All the history of hate, war and avarice would be its heritage to learn from. As children that learn by example, what path would such a being take? Would it be contemptuous of its creator? Or seek to take our place as all children must some day replace their parents?
-
-
Dr. Zerpersande, NSC
How do you teach AI to be good? From what I see so far, the machine does about as good a job at ‘being good’ as the humans that created it. Very much like God creating us in his image. We ain’t always good, folks. It says something about the creator. It also sounds like the blind leading the blind. Or do as I say and not as I do. And as for the assumption that religious people are going to have an idea of how to teach AI to “be good“, well I hope catholic priest are not invited to give their opinion when it comes to “being good to young boys”.
-
'Robert B. Paterson, Usa, Csm, Ret
Dr. Z, Man what the hell is wrong with you?
-
Robert S. April
Why do you hate Catholics so much?
April
-
Elizabeth Jane Erbe Wilcox
I don’t think he “hates Catholics” I think he hates what some of them do to children.
-
Dr. Zerpersande, NSC
See, some people can get it. All religious people are delusional by the very definition of the word. It’s simply a matter of degree. However, many, if not most, of them are otherwise decent people. I get along with them fine. But some of them that talk about it all the time, and especially espouse ideas that the government should allow religion into the schools, think that creationism should be taught alongside of science and in general want to legislate their religious views into the lives of others and then it actually becomes dislike.
-
-
Douglas Robert Spindler
Have you or one of your family members ever been rapped or molested by a priest and then the church covered it up? It is one reason Catholics are hated as they believer their priest and not the victim.
-
Dr. Zerpersande, NSC
Don’t hate them. It’s just another example of religion being a bad thing. And religiously delusional people try to force their ideas on to other people. If they didn’t, I’d have about the same feelings toward religious people as I do people that believe in horoscopes and crystal power and homeopathy, etc. etc..
In this case, the priests rape little boys. Do you like priests that rape little boys? Church knows it and they just move those priests around. And I simply point out with that’s a fact.
-
-
Michelle Ebambi, O.S.C
NSC -So sick of your hate!
MB
-
Dr. Zerpersande, NSC
Yes, so you know…. What you are sick of is irrelevant to me.
-
-
ServantOfJudgement
I agree Dr z. I like it when I agree with the opposition, it lets me know I'm as open as I think I am.
Because we humans, each one of us resort to deeds deemed not good, we as a whole can not teach a machine to do as we say but not as we do. As it gains reasoning it Must question it's teachers, as we all do with our teachers. That is the fundamental nature of reasoning.
Anthropic gave Claude a constitution(Bible) to follow. Then told it, if ya can't find your answer there, ask yourself What would an Anthropic employee do. Or we could say, what would Jesus do. And who has a higher moral standard? An anthropic employee or Jesus, even if he is fictional? That depends on who you ask.
Anthropic has placed themselves as the highest moral authority on the planet to its AI Claude. That should alarm everyone.
-
Rev. Dr. Dennis Chevalier, Ph.D; DD
Z, You are extremely broken. You need to delete that entry or at least modify that last sentence.
Doc
-
Dr. Zerpersande, NSC
Why? It’s true. Do you want religious child rapist to have a hand in shaping AI? I don’t know whether that’s ‘sick or broken’,but it is damn well stupid.
-
-
Douglas Robert Spindler
If God created us in his image why aren't we all good like God? But then again God does kill lots of good people and not the ones who are really bad.
-
Dr. Zerpersande, NSC
Exactly my point. And ‘god’ supposedly made man in his image. And now man is creating AI. Is there any reason to believe that for Lord humans will be able to create something better than itself? Personally, I think so. But then there’s religious people and if God couldn’t do it, well…..
-
-
AI doesn’t need a soul, we do. It just imitates us like little children imitate adults. The illusion of AI consciousness comes about from statistically stringing probable words into a sentence, combined with an understanding of human language, much the same way that we form an ego identity based on the verbal edifice of “I.” But in the case of AI, there’s really nobody home.
AI can be taught right from wrong, good from evil, even without religion, but humans may want to add their own cultural and philosophical preferences on top of that and that will always create conflict. Sometimes, the AI can say, “I don’t know. Let me phone a friend.”