Die Financial Times hat eine richtig gut gemachte Page, auf der interaktiv erklärt wird, wie ChatGPT funktioniert, was ein Large Language Model (LLM), ein Token oder das Transformer Model ist, wieso man nicht wirklich von künstlicher Intelligenz, sondern mehr von einer Art Maschine, die Texte auf Basis hochgradig perfektionierter statistischer Wahrscheinlichkeiten erstellen kann, wenn man über ChatGPT spricht.
The LLM is underpinned by a scientific development known as the transformer model, made by Google researchers in 2017. […] First a block of words is broken into tokens — basic units that can be encoded. […] In order to grasp a word’s meaning […] LLMs first observe it in context using enormous sets of training data, taking note of nearby words. […] A word embedding can have hundreds of values, each representing a different aspect of a word’s meaning. Just as you might describe a house by its characteristics — type, location, bedrooms, bathrooms, storeys — the values in an embedding quantify a word’s linguistic features. The way these characteristics are derived means we don’t know exactly what each value represents, but words we expect to be used in comparable ways often have similar-looking embeddings. […] But this alone is not what makes LLMs so clever. What unlocked their abilities to parse and write as fluently as they do today is a tool called the transformer, which radically sped up and augmented how computers understood language. Transformers process an entire sequence at once — be that a sentence, paragraph or an entire article — analysing all its parts and not just individual words. This allows the software to capture context and patterns better […] The transformer has resulted in a host of cutting-edge AI applications. […] It drives autocomplete on our mobile keyboards and speech recognition in our smart speakers. Its real power, however, lies beyond language. Its inventors discovered that transformer models could recognise and predict any repeating motifs or patterns. From pixels in an image, using tools such as Dall-E, Midjourney and Stable Diffusion, to computer code using generators like GitHub CoPilot. It could even predict notes in music and DNA in proteins to help design drug molecules. For decades, researchers built specialised models to summarise, translate, search and retrieve. The transformer unified all those actions into a single structure capable of performing a huge variety of tasks.
Financial Times
Das ist der Artikel für Freundinnen und Freunde des Scrollens und des Betrachtens von durchs Scrollen ausgelösten Animationen. Ein kurzer Einblick, was da eigentlich passiert, aber ein Augenöffner für diejenigen, die verstehen wollen, wieso ChatGPT und andere KIs das leisten können, was sie leisten.
Wer es etwas genauer haben will und vor ein ganz klein wenig Mathematik, Tabellen und Diagrammen nicht zurückschreckt, sollte sich aber im Anschluss Stephen Wolframs Artikel „What is ChatGPT doing… and why does it work?“ durchlesen.
That ChatGPT can automatically generate something that reads even superficially like human-written text is remarkable, and unexpected. But how does it do it? And why does it work? My purpose here is to give a rough outline of what’s going on inside ChatGPT—and then to explore why it is that it can do so well in producing what we might consider to be meaningful text. […] The first thing to explain is that what ChatGPT is always fundamentally trying to do is to produce a “reasonable continuation” of whatever text it’s got so far, where by “reasonable” we mean “what one might expect someone to write after seeing what people have written on billions of webpages, etc.”
Stephen Wolfram
Ich denke, ein wenig Wissen über Künstliche Intelligenz – oder zumindest über das, was wir, eigentlich fälschlicher Weise, als solche bezeichnen – ist im Jahr 2023 Allgemeinbildung. Also lest, lernt, wisst!