What Is Generative AI and Why Is It Important?
Jun. 19, 2023The age of artificial intelligence is here, and Generative AI is playing a pivotal role in bringing unprecedented advancements to everyday technology. There already are severalfree AI toolsthat can assist you in generating incredible images, texts, music, videos, and a lot more within a few seconds. But, what exactly is Generative AI and how is it fueling such rapid innovation? To learn more, follow our detailed explainer on Generative AI.
Definition: What is Generative AI?
As the name suggests, Generative AI means a type of AI technology that cangenerate new contentbased on the data it has been trained on. It can generate texts, code, images, audio, videos, and synthetic data. Generative AI can produce a wide range of outputs based on user input or what we call “prompts“. Generative AI is basically a subfield of machine learning that can create new data from a given dataset.
If thelarge language model (LLM)has been trained on a massive volume of text, it can produce legible human language. The larger the data, the better will be the output. If the dataset has been cleaned prior to training, you are likely to get a nuanced response.
Similarly, if you have trained a model with a large corpus of images with image tagging, captions, and lots of visual examples, the AI model canlearn from these examplesand perform image classification and generation. This sophisticated system of AI programmed to learn from examples is called a neural network.
That said, there are different kinds of Generative AI models. These are Generative Adversarial Networks (GAN), Variational Autoencoder (VAE),Generative Pretrained Transformers (GPT), Autoregressive models, and much more. We are going to briefly discuss these generative models below.
At present,GPTaka Transformer-based models have gotten popular after the release ofGPT-4o/ GPT-4 / GPT-3.5 (ChatGPT), Gemini 1.5 Pro (Gemini), DALL – E 3, LLaMA (Meta), Stable Diffusion, and others. OpenAI also demoed itsSora text-to-videomodel recently.
All of these user-friendly AI interfaces are built on the Transformer architecture. So in this explainer, we are going to mainly focus on Generative AI and GPT (Generative Pretrained Transformer).
Amongst all the Generative AI models, GPT is favored by many, but let’s start withGAN (Generative Adversarial Network). In this architecture, two parallel networks are trained, of which one is used to generate content (called generator) and the other one evaluates the generated content (called discriminator).
Basically, the aim is to pit two neural networks against each other to produce results that mirror real data. GAN-based models have been mostly used for image-generation tasks.
Next up, we have theVariational Autoencoder (VAE), which involves the process of encoding, learning, decoding, and generating content. For example, if you have an image of a dog, it describes the scene like color, size, ears, and more, and then learns what kind of characteristics a dog has. After that, it recreates a rough image using key points giving a simplified image. Finally, it generates the final image after adding more variety and nuances.
Moving toAutoregressive models, it’s close to the Transformer model but lacks self-attention. It’s mostly used for generating texts by producing a sequence and then predicting the next part based on the sequences it has generated so far. Next, we have Normalizing Flows and Energy-based Models as well. But finally, we are going to talk about the popular Transformer-based models in detail below.
So what was the key ingredient in the Transformer architecture that made it a favorite for Generative AI? As the paper is rightly titled, itintroduced self-attention, which was missing in earlier neural network architectures.
What this means is that it basically predicts the next word in a sentence using a method called Transformer. It pays close attention to neighboring words to understand the context and establish a relationship between words.
Through this process, the Transformerdevelops a reasonable understandingof the language and uses this knowledge to predict the next word reliably. This whole process is called the Attention mechanism.
Coming to the “pretrained” term in GPT, it means that the model hasalready been trainedon a massive amount of text data before even applying the attention mechanism. By pre-training the data, it learns what a sentence structure is, grammar, patterns, facts, phrases, etc. It allows the model to get a good understanding of how language syntax works.
Both Google and OpenAI are using Transformer-based models in Gemini and ChatGPT, respectively. However, there aresomekey differencesin the approach. Google’s latest Gemini uses a bidirectional encoder (self-attention mechanism and a feed-forward neural network), which means it weighs in all surrounding words.
It essentially tries to understand the context of the sentence and thengenerates all words at once. Google’s approach is to essentially predict the missing words in a given context.
In contrast, OpenAI’s ChatGPT leverages the Transformer architecture to predict the next word in a sequence –from left to right. It’s a unidirectional model designed to generate coherent sentences. It continues the prediction until it has generated a complete sentence or a paragraph.
Perhaps, that’s the reason Gemini is able to generate texts much faster than ChatGPT. Nevertheless, both models rely on the Transformer architecture at their core to offer Generative AI frontends.
We all know that Generative AI has a huge application not just for text, but also for images, videos, audio generation, and much more.AI chatbots like ChatGPT, Gemini, Copilot, etc. leverage Generative AI.
It can also be used forautocomplete, text summarization, virtual assistance, translation, etc. To generate music, we have seen examples likeGoogle MusicLMand recently Meta releasedMusicGenfor music generation.
Further, Generative AI has applications in 3D model generations and some of the popular models are DeepFashion and ShapeNet.
Not just that, Generative AI can be of huge help indrug discoverytoo. It can design novel drugs for a specific disease. We have already seen drug discovery models like AlphaFold, developed by Google DeepMind. Finally, Generative AI can be used for predictive modeling to forecast future events in finance and weather.
While Generative AI has immense capabilities, it’s not without any failings. First off, it requires a large corpus of datato train a model. For many small startups, high-quality data might not be readily available. We have already seen companies such as Reddit, Stack Overflow, and Twitter closing access to their data or charging high fees for the access.
Recently, The Internet Archivereportedthat its website had become inaccessible for an hour because some AI startup started hammering its website for training data.
Apart from that, Generative AI models have also been heavily criticized for lack of control and bias. AI models trained on skewed data from the internet can overrepresent a section of the community. We have seen howAI photo generatorsmostly render images in lighter skin tones.
Then, there is ahuge issue of deepfake videoand image generation using Generative AI models. As earlier stated, Generative AI models do not understand the meaning or impact of their words and usually mimic output based on the data it has been trained on.View this post on InstagramA post shared by Beebom (@beebomco)
A post shared by Beebom (@beebomco)
It’s highly likely that despite best efforts and alignment, misinformation, deepfake generation, jailbreaking, andsophisticated phishing attemptsusing its persuasive natural language capability, companies will have a hard time taming Generative AI’s limitations.
Passionate about Windows, ChromeOS, Android, security and privacy issues. Have a penchant to solve everyday computing problems.
