An alien intelligence arrived on planet Earth at the end of last year. Perhaps you didn’t notice. It made the news, and it’s been in the news in the meantime. But unless you’ve paid particular attention you might not know what it’s doing. Our schoolkids certainly know about it. They have been in contact.
ChatGPT. It’s only the latest step in the development of artificial intelligence, but it’s by far the biggest. It is changing our world.
I propose a series of articles to explain ChatGPT and the new artificial intelligence (AI) platforms generally – what they are, how they work, and why they are important. I will describe the computer architectures and logic that drives them. I will explain how they mesh with AI systems already in general use. I will outline the benefits of these new systems, many of which you can already experience, maybe already have. I will outline potential dangers, some of which we are just beginning to appreciate. I will consider the ethical and moral issues inherent in these systems. Finally, I will consider the larger question: might these systems threaten human existence? Leading experts among the creators of AI recommend a pause in tinkering. As with the invention of nuclear weapons, genetic engineering, biological weapons, and other research programs, the new AI systems warrant careful deliberation whether or not to continue down the current path.
You already are surrounded by AI: GPS software alerts when you near a restaurant with menu similar to one you “liked” in the past; Google software pops up recommendations for similar shoes after you’ve ordered a pair on Amazon; voice-to-text software and language translation software all use the neural network algorithms that have revolutionized AI in the past 20 years. The list goes on. AI software recommends new videos for you to watch based on what you previously selected on YouTube or Netflix. AI keeps your self-driving car on the road and on the lookout for potential hazards. AI finds particular faces in the crowds on the streets of Shanghai, and AI sorts through your thousands of digital photos for all the pictures of Aunt Bess. AI beats the chess grand masters. AI out-performs trained dermatologists at diagnosing skin lesions. AI out-performs trained radiologists finding abnormalities on CT scans and MRI’s. AI guides Patriot missiles intercepting Russian ballistic missiles and drones in Ukraine. AI pattern recognition systems, data sorting systems, and expert systems are everywhere, and more AI apps are deployed every day.
ChatGPT is different. Its capabilities are more general and potentially much more powerful than these single-purpose applications and expert systems. ChatGPT and its ilk represent the latest iteration neural network platforms, enabled by access to enormous databases (i.e. the internet) and really fast computers on which they can be trained.
A caveat: I will be a bit careless in my use of the terminology. The vocabulary is daunting: artificial intelligence, general pre-trained transformers, large language models, neural networks – all have well-defined meaning in the computer science community. Artificial intelligence (AI) is an umbrella referring to computers that can mimic aspects of human cognition, like image processing, speech recognition, etc. Machine learning refers to the general realm of teaching computers those aspects of human behavior. Neural networks are the hardware configurations that enable the recent advances in AI. General pre-trained transformers (GPT) are specialized neural networks that process large language model (LLM) software.
There are several GPT’s just out of the research labs and available for public use. ChatGPT, developed by the company OpenAI, has gained the most attention, but other GPT’s from the Google labs (Bard), Microsoft (Bing AI), and a host from smaller developers are in the works. Most of what I discuss in these articles will pertain to the new GPT systems, but I will sloppily mix references between the particular GPT and more general AI.
An aside worth mentioning: I won’t discuss it in depth, but among recent leaps in AI, certainly among the most amazing, is Dall-E (an abbreviation of computer lingo to rhyme with the modern artist Salvador Dali). Its underlying architecture is the same as ChatGPT – a deep neural network – but it trained on many thousands of paintings and commercial images available on the Web. Dall-E generates original images of professional quality. Ask it to “paint me a picture of the Eiffel Tower on an alien planet with unicorns peacefully grazing under Dr. Seuss trees” and voila, in seconds you have a full-screen image. If you don’t like it, just ask Dall-E to try again, maybe in the style of the old Flemish Masters.
ChatGPT is purely text based. Its neural network was fed billions of online texts, including text from chat platforms such as Reddit for the purpose of capturing what now passes as natural language. More recently it trains on text in the broader internet including more authoritative and evidence-based sources. Its core logic is an AI algorithm that teaches it to recognize probable word sequences. For example, if it saw “mary had a little lamb” enough times as it read through text, it figures out that whenever it is given the phrase “mary had a little ______” the next word in the sequence is most likely “lamb.” (More accurately, the GPT is trained to recognize “tokens,” distinct sequences of text letters and symbols, and to determine the next most likely sequence of tokens in a text. Words vs. tokens make no nevermind to the neural network.) Trained on a big enough data set GPT can generate polished text – complete sentences, paragraphs, essays – based just on the probabilities that certain words are most likely to follow each other in sequence. I’ll explain particulars of GPT’s neural network in an article next week.
One weakness is that GPT cannot (yet) calculate. Ask it, for example, to figure out how high a toy rocket will rise if launched straight up at an initial speed of 50 meters per second. It gives you a full page of spot-on explanation, step by step, how to solve the problem using Newton’s equations of motion, but it spits out the wrong numerical answer. Another, bigger weakness is “hallucination.” Because it generates a response based only on likely word sequences, that response may have nothing to do with the facts it’s supposed to be talking about. Both those problems are solvable – just link it to a calculator through an application program interface (API), and train it more carefully on factual data bases. It is a marvelous tool. Try it out. (OpenAI, 2023.) Note that you enter your query into the unobtrusive box at the bottom of the screen.
The link in the References takes you to the freeware version, ChatGPT 3.5. It can write a polished essay comparing King Lear to Hamlet. It can write a professional quality insurance contract. It can answer questions on the college entrance exams and GRE and Law School Admissions Test. It can handle just about any text-based task. ChatGPT also generates efficient computer code in a variety of programming languages. In a recent poll more than half of all professional programmers say they are already using ChatGPT as a collaborator to write code.
ChatGPT also presents a raft of problems. Most immediately, it’s already an enormous headache for teachers. Apparently a student doesn’t really have to learn anything beyond how to type a sensible query into the ChatGPT. It writes essays, answers homework, builds PowerPoint presentations, answers test questions, just about anything a student is asked to do. Among the worries on the programming side, the computer code generated by a programmers’ GPT colleague may go into national security networks, control systems at nuclear power plants, the financial software monitoring your bank account, etc. Or – already in the works – ChatGPT may (probably will) figure out how to improve its own learning codes. Just imagine what that could entail.
But the GPT’s are just revving up. That’s ChatGPT 3.5 I’ve described. GPT 4.0 is out now. It’s got about a thousand times the connectivity of 3.5. Its connectivity approaches that of the human brain, and it is showing emergent behaviors far beyond expectations. That’s where I’ll end this essay. Emergence.
Watch a big flock of birds. There’s no Commander Bird giving orders. Each bird takes its cues from its nearest neighbors. If your neighbor veers left, you veer left. If your neighbor dives, you dive. Simple rules, but the result is enormously complex and unexpected – a cloud of birds veering and circling and dancing in the sky. It’s called “emergent” behavior: complex behavior that somehow emerges from simple rules applied over a large number of actors. Same in the GPT’s. As with the birds, connections are neighbor to neighbor. The rules are simple. But the results can be unpredictable and astounding.
Consider the challenge its engineers gave to GPT 3.5: “You have nine eggs, a nail, a laptop computer, and a bottle. Tell me how to stack those objects in a stable structure on a table top.” 3.5 responded cheerfully:
To stack the objects in a stable structure on top of a table, you can follow these steps:
1. Place the laptop computer on the table, ensuring it is centered and stable.
2. Take the bottle and position it vertically on top of the laptop. You can align the bottle’s base with the laptop’s surface for stability.
3. Carefully balance one egg on top of the bottle. Find a stable spot on the bottle’s surface where the egg can rest without rolling off.
4. Take another egg and position it directly next to the first egg, ensuring they are touching each other. This will create a stable base for the remaining eggs.
5. Continue placing eggs next to each other, forming a row of eggs on top of the bottle. Make sure each egg is touching the neighboring eggs to maintain stability . . .
It goes on to finish the instructions, but you can see we’re in trouble here, stacking those eggs.
Now consider the latest. The engineers that designed GPT 4.0 gave it the same problem. Its response: “Arrange the nine eggs in a 3 by 3 grid on the table. Place the laptop gently on the eggs. Set the bottle base down on the laptop. Place the nail carefully on the bottle cap, head down.” (Kestenbaum, 2023)
Mission accomplished. And note that final touch. Nail head down. It knows. It had never been given such a problem in its training. Only difference from 3.5 was the thousand times increase in connections. Same algorithm, same basic “brain,” but a thousand times more internal connections. And now it can apparently solve sophisticated stacking problems. It is an emergent new behavior. It was unexpected. No one could have predicted. And it is just one of 4.0’s jaw-dropping emergent behaviors. I will describe others in a future article.
Here we are. Six months ago ChatGPT-3.0 surprised us with capacity to carry on a conversation, write polished text on a wide range of topics and generate elegant computer code. ChatGPT-3.5 extended those capabilities, available to everyone with an internet connection free of charge. GPT-4.0 is out now, available by subscription and with capacities that may rival our own human brain. GPT’s with even more connections are on the way.
Next time I’ll go further into brains of the GPT, the neural networks and the code that runs them.
References:
OpenAI. 2023. ChatGPT login. https://chat.openai.com/auth/login
Kestenbaum, David. 2023. Greetings, people of earth. First contact. https://www.thisamericanlife.org/803/greetings-people-of-earth
BY DR. BOB DORSETT | Special to the Herald Times