AI is hot again. It feels like a flashback to 2015. But this time, it is different. AI is finally moving into its industrial age. Everyone who tried ChatGPT knows it.
Machines now not only know more than a human possibly can, but can also be nearly as creative. But only nearly... Here is the reply my colleague Michael got when he asked ChatGPT about the outcome of the Fifa World Cup ahead of the finals 😀:
'I'm sorry, but I am unable to predict the future. My knowledge is limited to the information that was available to me at the time of my training, which ended in 2021. As a language model, I do not have access to current events or the ability to browse the internet, so I am unable to provide information on future events such as the 2022 World Cup. Is there anything else I can help with?'
One paragraph in this post has been written entirely with ChatGPT. At the end of this post, I will tell you which one...
What is all the hype about AI all of a sudden?The magic of ChatGPT is that it gives mass market access to generative AI through a nice user interface. You type any question and get a remarkably accurate answer: from entire essays (students around the world are cheering) to detailed research data (find the paragraph in this blog post) or speeches for special occasions (my colleague Marina gave to my team last week a Christmas speech written in a couple of seconds using OpenAI's tech).
What is coming is so much bigger than just OpenAIs LLM (language learning model). AI models did huge strides in the last five years. You might not realize it, but you likely interact with some form of language model every day already: it is used for spam detection, autocomplete, and it enables you to talk to Siri & Alexa, amongst many other applications. But Deep Neural Networks have been around for about 70 years. It's taken a long time to get here. The basics haven't changed but what has are: GPU's, much more Data (and data creation is reeaally taking off, 50x expected growth over the next 15 years according to IDC… more on this in a later post), critical recent advances in AI models, and solid state drives. The chart below gives you an idea of the estimated computing resources used. Google's PaLM, for example, boasts 540B parameters (including a big chunk of the internet)... The biggest model size for the soon-to-be-released GPT-4 will be 1T parameters - up 6x from GPT-3 and 500x more powerful.
The number of parameters in a model determines the model's capacity and ability to learn from data. In general, a model with more parameters will be able to learn more complex patterns in the data and will perform better on tasks that require a deeper understanding of the input data. However, a model with too many parameters can also suffer from overfitting, where it performs well on the training data but poorly on unseen data.
The infrastructure layer giants.All of this led to the rise of these so-called foundation models. 'These are large neural networks 'pre-trained' on massive amounts of data without particular end-uses in mind'. OpenAI’s GPT-3 (175B parameters dataset) and Google's Palm (540B parameters) are two of those examples. They bring the incredible powers of AI to businesses by giving API access. The key is that these foundation models can then be 'fine-tuned' on smaller sets of hand-labeled data to perform specific tasks, like answering customer questions. You must check out this video my partner Mark shared with me on specific task automatisation if you want a glimpse of what is coming. This new class of large models, also labeled Generative AI, makes it possible for machines trained on specific datasets to write, code, draw, etc. and create human-level results in seconds. Here a 'drawing in black, yellow, and white of the world cup' by Stable Diffusion. Right now, it would still take me too many iterations to do the illustration of my blog with this tech, for example, but if fine-tuned just a bit, it could save me a ton of time and, finally, just do the perfect job for me.
On the foundation level, we will witness the emergence of only a handful of massive companies that will dominate this infrastructure layer. The reason why there will only be a few is simple: the cost to run these infrastructures is enormous. ChatGPT is costing OpenAI an estimated $3M a day to run... Similar to the rise of cloud vendors a decade ago, this new class of companies will specialize in training and maintaining the latest up-to-date models. Their massive scale drives enormous value. Like for cloud vendors, the product will become somewhat commoditized over time and they will operate some sort of pay-by-volume business model.
The application layer race is on.It goes without question that, with the platform layer solidifying, every piece of software will leverage these models in their core components. This automatisation will not only increase productivity, but give rise to a new generation of businesses that will enhance these pre-trained general models by focusing on specific use cases or narrower tasks. These are fascinating days for the tech world. Many of these new-gen businesses are already out there. If we just look at the mobile and how the back then new capabilities such GPS, 24/7 connectivity, and cameras gave rise to new massive businesses, one can only dream of how this new technology will impact the tech world (see chart above).
We are still very early days, though. ChatGPT showed all of us that it is very real, but also, that a lot of fine-tuning still has to be done. It is no surprise that Google won't launch a ChatGPT competitor (yet) as the tech is still too immature to put in front of its users. However, the internet giant is worried about the leaps its competitors have made, searching for answers (pun intended) internally. In other words, the platform layer is getting good, and the application space has barely gotten going. The race is on.
How will this impact software going forward?After playing with some of the applications out there, here are some of the areas where I think we will see an impact in the relatively short term:
1. Content Creation become ubiquitous.Everyone will soon be able to produce any type of content. Models are doing quick progress in all content formats, in language, audio, video, code, etc. This will be a paradise for deep fakes... Just check out this fake podcast between Joe Rogan and Steve Jobs. The quality is absolutely mind-boggling. Teachers worldwide are already complaining about students using ChatGPT to write their essays and do other homework. On the flip side, we will discover more stunning digital experiences and visualizations across the web.
'Hand-crafted' digital content will be comparable to artisanal physical goods sold on Etsy or in a local shop for example. We will have mass-market content 'made by AI'. On the other hand, there will still be blog posts like this one that are 'hand made'. Great communicators and writers will have an edge. People will leverage AI to be more productive, of course, just as an artisan uses industrial items in his production.
2. Everyone will have a verified digital identity.Since the early days of blockchain, I have been fascinated with its potential in the digital identity and verification space: passports, identity cards, educational diplomas, and so on.
Take the small European country Estonia as an example. Since 2003, it has been building the most sophisticated e-government in the world. Everything from taxes and transfers, incorporating a business, paying bus fare, applying for citizenship, certifying a marriage, and tens of thousands of other services can be performed electronically. It’s then all saved on a distributed, cryptographically secured ledger (developed in the 2000s) as a precursor to the blockchain. Many systems are thus automated, like auto-enrolling your kid in school four or so years after filing their birth certificate.
Content 'made by AI' will now reverse the power dynamics. Instead of waiting for governments to roll out tech top-down, users will start using the tech to verify their digital content and the push will come bottoms-up. Otherwise, how do we know whether content is original or trustworthy? Blockchain addresses seem to be a handy solution to verify the originality of digital content. OpenAI is already working on a way to 'watermark' AI-generated content...
3. Search will be proactive and instantaneous.AI models are creating a new era for search. By search, I don’t just mean internet search like Google - but the ability to query information and ultimately synthesize conclusions. Here is an excellent example of a question by Martin: 'How many people could live in the world if the whole world (minus Antarctica) would have the density of the Netherlands. Answer 74 billion. We are now 8 billion. The Netherlands is the world's 2nd largest food exporter...'
Search is a trillion-dollar opportunity that spans consumer, enterprise, and developer ecosystems. This definition encompasses everything from enterprise document search to conversational consumer products. AI that finds the needle in the haystack will be big.
4. AI, including voice, will become the primary software interface.One can easily imagine Natural language becoming the primary interface to software apps. In 2020 I claimed that 'the best interface is no interface'. In this report, we even predicted the death of the keyboard in 5-10 years. I think OpenAI showed us all that this prediction will become a reality soon.
5. Processes will be automatised across the board.Processes will be optimized across the board. The products of companies like UIPath will improve even further and many new specialized productivity tools will see the light of day.
One great example is how Generative AI allows generating code by writing what you want to build with simple words GitHub's Copilot is being used by more than a million developers already helping them be more efficient!
Another example is 'how models can be used to generate new molecules and medicine that humans didn’t think of or couldn’t take the time to test. Microsoft developed a model that seems very promising in this area'.
6. Cybersecurity will be a top challenge for everyone.Cybersecurity is already one of the key challenges of enterprises and SMEs globally. API access to foundation models will lift hackers to an entirely new level. Encryption techniques such as Zero Proof Knowledge will become even more sought-after.
These are just some examples, of course. When I asked a friend last week what he thinks about all of this, he just replied: 'Man, it feels like in real life experience just got so much more valuable. Authenticity will probably be the killer feature in the years ahead.' 😉
Last week I was in Paris and had the chance to watch the world cup game of Morocco against France in a small Pizzeria. It was terrific to see Moroccan and French supporters sitting next to each other, cheering for their teams, and just having fun together. Despite some fears, the atmosphere in the city was overall excellent post-game. This is great, and as it should be. Europe has a great future.
A Merry Christmas to all of you!
Life is awesome,
Yannick
P.S. The paragraph starting 'The number of parameters' was written entirely by ChatGPT. Did you get it right? 😀
Other content I found useful.- A cool report by Dealroom on the next generation of tech ecosystems. This report is the conclusion of a data-driven analysis of 201 tech ecosystems.
European VC Europe