A language is a set of sounds for which there is a social pact governing meaning and grammar. There are two ways to learn a language. The first is as an adult learns a foreign language by studying its grammar and vocabulary. The second is as a child learns by imitating others.
ChatGPT uses the latter method. It scans a humongous amount of literature on the web, and takes iterative feedback from its trainers. Each of the 50,000 commonly used words in English are allotted a number. At any given point, it looks for the most common word which usually follows the text it has already generated. These words each have a context-based probability of occurrence. The next word is chosen using a degree of randomness. This process continues until a complete essay has been generated.
For creative work, the randomness must be higher. To get more precision, a lower level of randomness is needed. If ChatGPT were to always take the word with the highest probability, then its output would be monotonous and not have any human-like quality. Note that the essay is not reviewed. The process used is strictly unidirectional. Also, instead of actual words, ChatGPT uses word tokens, or parts of words. This is why it can sometimes create new words.
On a technical level, ChatGPT is basically a giant neural network. This technology is not new; it has been around since the 1940s. Advances in computing power have made it practical to use neural nets in a productive manner. Note that neural nets try to mimic the workings of the human brain. Software engineers don’t give specific instructions to AI and tell it what kind of text to generate. Instead, they code the rules which govern how the network makes decisions.
When given a prompt, ChatGPT does not go out and scan the web. It has already done that once, and created a dataset which represents probabilities of occurrence of each word in a given context. These probabilities are modified during training, with outcomes favored by the trainers getting higher weights. This dataset of probabilities and weights is used by the neural net to generate word tokens in an iterative manner.
The generic dataset of ChatGPT can be customized by adding a layer representing organizational policies, such as HR, Legal, IT, etc. The resulting product can be used by employees to get answers to frequently asked questions. To get consistent results, a lower degree of randomness must be used. This customized dataset would represent the key to an organization’s business secrets and would need to be protected.
ChatGPT, or AI in general, is not going to replace humans. It is not going to take over the world and destroy humanity. The reasons for this are partly technological, and partly philosophical. Dwelling on them in more detail here would take us off course. However, this should not lull us into underestimating the impact of AI. It is going to geometrically increase productivity and create whole new industries.
In the short term, this will mean job losses to some extent. We should not fear this. To put this in perspective, the advent of computers did not lead to large scale job losses, destitution, and riots. We simply transitioned, quite smoothly, into a far more productive world. This has been going on for a very long time, from the invention of power looms hundreds of years ago. Our best approach would be to embrace the new technology, and avail ourselves of the opportunities it provides.