What is GPT-3?
GPT-3 (Generative Pretrained Transformer 3) is a third-generation language module developed by OpenAI was developed and based on Natural Language Processing (NLP)-models. It is the predecessor model to GPT-4.
The company, which was co-founded by Tesla CEO Elon Musk, is engaged in the development of Open source solutions in the area of artificial intelligence and has set itself the goal of working out the advantages of this for humanity. For the founders, as well as some scientists, there is a danger of human intelligence being surpassed or replaced by artificial intelligence.
Compared to its predecessors GPT-1 and GPT-2, the current version, which was introduced in May 2020, has achieved several improvements. The first version, GPT-1, was an NLP model that, in contrast to the previous state of the art, did not have to be trained specifically for a particular task, but only required very few examples for high-quality speech output. OpenAI staff further developed this model by expanding the data set behind it, adding more parameters and thus creating GPT-2.
This language module also has the ability to understand instructions as such, for example, to translate texts automatically by text instruction. While GPT-1 and GPT-2 are freely available as open source software, a commercialisation of the product took place with the GPT-3 language model. OpenAI argues this move on the grounds that freely distributing the new version poses too great a risk of spreading misinformation and spam or fraudulently writing academic papers due to its strong performance.
How does the language model work?
Compared to its predecessor, the third version takes into account a hundred times more parameters and uses five data sets (Common Crawl, WebText2, Books1, Books2 and Wikipedia), while GPT-1 (Book Corpus) and GPT-2 (WebText) only use one each. Training data set accesses.
The basic idea of many language modules in generating texts is to use statistical models to predict the next words in a text so that the text makes grammatical and linguistic sense. The AI software GPT-3 does not work with words or letters, but with so-called tokens. Basically speaking, this is a sequence of characters that belong together. In this way, the GPT-3 speech AI manages to bring variance into the speech output, which would be more difficult to represent by looking at whole words.
The language module applies the following models (so-called engines) for the analysis and generation of texts: Davinci, Curie, Babbage and Ada. Each offers advantages and disadvantages for certain areas of application. While Davinci is suitable for the analysis of complex texts, Curie is suitable for the use of a service chatbot. The user specifies this and a few other parameters to determine the output. Such parameters are, for example, the creativity of the output text as well as its length.
The language AI GPT-3 was developed for the English language and can currently only develop its full potential in this language, even though it offers translation options.
In which software is GPT-3 used?
Basically GPT-3 can currently be used via API interface and corresponding payment. Microsoft, on the other hand, has secured the exclusive licence to the source code of GPT-3 in order to integrate the language module into its own products and develop it further. For example, Microsoft Azure OpenAI combines the speech AI with the Microsoft services of security, access management and scaling capacity. This overall solution is said to be of particular interest to companies.
Furthermore, it is possible to GPT-3 for example also in Chatbots so that the software helps to conduct conversations and offer assistance. The best-known example of the use in chatbots is the Microsoft-owned service ChatGPT. Furthermore, it is also used in game development to create automatic dialogue and storylines in computer games. Due to the engine parameterisation, entire marketing texts, landing pages or newsletters, for example, can also be created and implemented on the basis of a small amount of information.
