AI 2023 Recap + 2024 Predictions with Rob Toews, Partner at Radical Ventures

This is Cross Validated, a podcast where we speak to practitioners and builders who are making AI deployments a reality.

Today, we have a special episode with Rob Toews, Partner at Radical Ventures, a VC firm focused on AI. He is also a contributor to Forbes, where he writes about the big picture in AI, and recently spoke at the Ted AI conference. Rob is a good friend whom I talk to a lot about the shifting AI landscape. And given how dynamic 2023 has been in the world of AI, I thought it would be a good way to wrap up the year by talking about some of the key trends in 2023 and some predictions for 2024. Enjoy!

Listen and subscribe on Spotify and Apple.

Follow me on Twitter (@paulinebhyang)!

Transcription of our conversation:

Pauline: Welcome to Cross Validated, a podcast with practitioners and builders who are making AI in the enterprise a reality.

I'm your host, Pauline Yang. I'm a partner at Altimeter Capital, a lifecycle technology investment firm based in Silicon Valley. Today, we have a special guest, Rob Toews, who's a Partner at Radical Ventures, an AI focused venture capital firm based in the Bay Area and in Toronto. He's also a contributor to Forbes, where he writes about the big picture in AI, spoke at the recent TED AI conference, and is generally a thought leader in AI.

And this is a really special episode in that normally we interview practitioners and ask them how their companies are using AI, but 2023 has proven to be such a dynamic year in AI, but I thought it would be a special end of year episode to bring on someone that I talked to about AI a lot, which is Rob and just discuss what's happened in 2023 the early signs of trends that we've seen and talk about 2024 some of the predictions that we have and reflect on all the changes that's happened this year and so thanks, Rob, for coming on to the podcast. I'm really excited about the conversation today. We'd love to just start with your quick background.

Rob: I'm very excited to be here. Looking forward to the conversation. My quick background, I'm a partner at Radical Ventures. Radical is a firm totally focused on AI. With close ties to top AI researchers like Geoff Hinton and others. I lead Radical’s Bay Area office and I joined Radical a few years ago from another VC firm Highland capital, where I've led Highlands investing for a few years. Before I got into VC, I worked in the world of autonomous vehicles for a long time.

Most recently, I helped lead the strategy team at Zoox, which was an autonomous vehicle startup based in the Bay Area that Amazon acquired a few years ago for a little bit over $1b dollars. And before Zoox, I did a brief stint in the world of policy. I worked in the White House toward the end of President Obama's second term, working on autonomous vehicle and AI policy and regulatory issues. I did grad school at Harvard and started my career at being in company and consulting. 

Pauline: I really appreciate that. Thank you. 

So I thought we'd start off with just a few stats on where 2023 funding trends have been. According to PitchBook, funding for AI related startups surpassed $69 billion in 2023 where OpenAI, Anthropic, Cohere, where I know Radical as an investor, accounted for a large portion of that number.

And so would love to just get your perspective on why you think the foundation model layer has attracted so much funding? And what do you think that means for setting up 2024 in terms of competition?

Rob: It has been a wild year indeed in early stage VC in the world of AI. You're totally right. As you said, the lion's share of venture capital dollars in AI continue to flow to companies building at the model layer: OpenAI and Anthropic and Cohere and others, as you mentioned.

And I think a lot of it is just well, to start with, there's the obvious reality that building models is just so capital intensive that the leading companies need to raise many billions of dollars, primarily to fund the computes to train these models. Current leading research methods to train high performing large language models and other foundation models is just incredibly, compute intensive.

And everyone's heard about the GPU shortage. None of these companies can get their hands on enough chips to train the models that they want. So a big part of it is just the kind of the stark economic reality of what it takes in 2023 to build the world's leading models, and I also think there's a bit of the oversimplified trajectory of new technology cycles is infrastructure buildout precedes application development, and we are certainly still in that infrastructure phase in the world of AI, where the core foundations are still being put in place to support a broader AI economy.

And the kind of the core intelligence that will power a lot of future applications is still being built. So the massive investment in foundation model builders is reflective of that. And I think, as those companies mature, we'll start to see more of an ecosystem of applications built on top of them. In terms of what the trend means for 2024 and kind of what the funding landscape looks like, I expect massive amounts of dollars to continue pouring into the model builders in 2024. I don't think that's gonna stop. I feel confident opening eyes going to raise another massive round in 2024.

Anthropic will raise more, Cohere will raise more, Inflection will raise more. Those round sizes will probably just continue to get larger. So I think that's not going to change. But I do think that the application layer is going to really mature and start to proliferate and a lot of interesting new directions.

 Like when LLMs first burst on to the scene a year and a half ago, there are so many possible applications so many ideas for ways you can use this technology that everyone has had and has talked about. But to this point, there really aren't that many great killer applications out there that are in use widely. And I think that just reflects the reality that it's hard to build applications on a new technology stack that get widespread adoption, especially in the enterprise. And so today I think coding is one application that already has gotten pretty widespread adoption with GitHub Copilot and others. But there aren't very many other than that.

So I do think that 2024 is going to be a big year for the application layer for companies that are not building their own models necessarily, but are taking models from OpenAI, for instance and others, and building applications on top of them that are not just the so called thin wrappers, but that are actually really thick complex products or platforms that create value in different ways using the underlying models.

Pauline: That was such a helpful framework for so many different things that I wanted to talk about today. And so let's dive into a few different areas starting with that this point that you made that you think the next year is going to bring even more funding to these small group of companies, independent companies, but in even larger scale.

One thing that's been really interesting to notice, I'd say the framework that I have for foundation model companies is similar to cloud providers and so the four big ones you call it are Azure within Microsoft, AWS within Amazon, GCP within Google and Oracle OCI within Oracle. And I think one thing that's interesting is that these are independent companies. I'm using air quotes for the audience can't see. I don't have the math as to how much of the $69 billion but so much of the funding has come from corporate and in particular these four companies plus NVIDIA, which is the main chip provider for this set of companies and what I've seen is sort of that, it seems like they're starting to find their niche. OpenAI really has focused on ChatGPT. Anthropic has more focused on the API layer especially through Bedrock. Inflection is focusing on Pi, which is the personal assistant. Cohere’s focusing more on the enterprise workflows.

How do you think that those differences will compound in 2024 as this sort of competition? Maybe in terms of number of companies won't get bigger, but each of these companies will just have more amounts of capital and have to figure out, do I spend more on just compute? Do I start building products? What's your thought on that? 

Rob: It is a really good question. I think the analogy to the cloud market structure is a helpful one. It's the right way to think about it. Cloud is not winner-takes-all. There's not one dominant provider, but there also is not and never will be hundreds of companies providing cloud services. It's just so capital intensive and expensive to stand up these data centers and keep them running year after year.

And I think providing foundation model capabilities has a similar dynamic. Likewise, I don't think it will ever be winner take all, but I also don't think there will be, certainly won't ever be hundreds. The exact number in the world  like, is it four? Is it six? Is it 10? Is it 12? It’s hard to say, I think it's likely there will be more LLM providers than there are big cloud providers just because there's so many different use cases and sources of differentiation but in terms of how that differentiation starts to happen in 2024, I'm honestly not sure how much specialization will happen within the next year. I mean, you mentioned that Inflection.

There are some foundational providers with leading LLM teams who have already the founding ethos is more oriented to a specific application. So Inflection you mentioned really is oriented around this chat bot personal companion persona for consumers. Character AI is another one that like they're building world class models team, but they're not trying to build with OpenAI building. They're trying to build the chatbot personal companion experience for consumers. So I think there are some like those, but among the kind of OpenAI cohort, I just think the market is so big and still so nascent that I would guess that there won't be really clear, sharp dividing lines by 2024.

 I guess, to zoom way out, like, basically everything that every business transaction, every interaction that humans have pretty much involves language and so much of that is going to be automated in the years ahead and so like the addressable market is so massive and there are so many different use cases within that.

I think they'll all remain pretty horizontal and will have success doing that. With that said, there obviously are differences between them and their value propositions. And Anthropic, for instance, has been and can continue to lean into their value proposition around really safe, responsible AI, this kind of unique constitutional AI approach for which will be very important for some customers and will compel them to go with Anthropic.

And Cohere has really leaned into this vision of being the most enterprise grade enterprise ready offering. And that will matter to some folks and OpenAI obviously has a ton of momentum and advantages in terms of being the most well funded team, the biggest team, the highest performing models. Currently, they're the incumbent in a lot of ways. So it will be interesting to listen to this in your front and see what happens, but my best guess is that they all stay fairly horizontal. 

Pauline: There's also a totally separate group of companies that we haven't talked about yet that I want to bring into the conversation, which is of course open source companies.

And you have, I'd say the probably arguably the leading open source model is Llama 2, which is from Meta. Mistral, which is a French company with researchers that have come out of Meta and Google DeepMind, actually just recently announced their Series A funding at a $2 billion valuation, and have recently also put out pretty impressive models around mixture of experts.

And so I think there's a broader question about what is the commoditization question around these models. What is the role that open source and closed source plays? And so how would you think about where we are in 2024 in terms of the commoditization of the models and the role that open source plays in in that ecosystem?

Rob: It's a great question. This is maybe I've made a bit of a spicy take or perspective on this, so I guess the different companies have different motivations and incentives here. So I think Meta is a very different beast when it comes to open source and some of the startups. I think Meta has basically decided very consciously to take the approach of   We don't have the very best models and that's not what the dimension we're going to compete on. So we're just going to open source really good ones and try to undercut the advantages that other folks have undercut OpenAI / Microsoft, undercut Google, which I think is a brilliant strategy. So, but I think Meta's not thinking about their models as a direct revenue driver for them so much as a way to equalize the playing field more, which I think is a really savvy way for them to play it.

The other open source model foundation model startups, like the Mistral's of the world, I think it's very unclear what the actual business model will look like for open source AI and I'm not sure there is a good one honestly. I think there's the most common narrative is you work with enterprises to help them deploy models. You customize the models for them, etc.

That was the narrative that Stability AI I had, which was last year’s darling open source foundation model builder and that really hasn't worked for them. And obviously Stability has had a lot of other problems as a company not directly related to their business model but I think Stability has shown that there's not a really clear compelling source of product market fit for open source AI models. And I just don't know if one exists.

And so I will be interested to see if Mistral continues to open source their very best models. I would not be surprised if they eventually end up moving in the direction of an OpenAI, Anthropic, etc. with more of an OpenAI in year where they are still open sourcing some powerful models but the very best ones the most advanced ones end up remaining proprietary and they monetize that way.

That seems to me like maybe the most possible path for them in terms of building a business and so in my view I think the best models will continue to be closed source. I think the gap between open source and closed source models will continue to persist. I think the very best models will continue to come out of places like OpenAI and Anthropic.

And again potentially even companies who currently position themselves as open source and maybe one related point to make on this. I think there is a interesting tension between a lot of the support and excitement and enthusiasm and investment theses for backing open source companies have like a very heavy philosophical underpinning to them.  

And I think for a very good reason, there’s a lot of enthusiasm about the importance and power of open source software more broadly and how important it's been over the decades. And that can kind of sometimes get intertwined with a bit of libertarian view of the world like software should be free and there shouldn't be any regulation and so forth, which again, I think that there's like a very credible view of the world, but enthusiasm for open source for those reasons isn't necessarily the same thing as enthusiasm for open source because it's going to be a great business. And so I think some of that inflation has happened and will be interesting to see play out. Obviously many folks are more bullish on open source of the commercial opportunity as well. So I could be totally wrong but that's kind of how I've thought about things.

Pauline: It is interesting. I've certainly been thinking about this a lot in that. Because Meta is a public company and has shareholders to answer to. Mistral has raised from more traditional venture capital firms. Investors want returns, and as we know your earlier point that the foundation model companies will continue to raise large amounts of capital is necessitated because the next model runs are going to be in the billions of dollars range.

I think our estimate for OpenAI GPT-5 is in the $2-3 billion range. And I think there is a question of as Meta continues to spend money on compute to train Llama 3 and Llama 4 and to your point, I agree with you that I think they're not expecting a huge revenue or any revenue return on that.

The question is at what scale of compute investments do shareholders say, this is not a good ROI for you to spend this much capital on this project that will never get you revenue potentially. And to your point, I think Mistral can always make the decision to close source their models their next models.

And so you mentioned also that you think that the gap between open source and closers will persist. Let me ask you more specific question:I think on Twitter, there was a great chart that showed that the gap right now is about a year. Going into 2024 do you think that gap of a year increases or decreases?

Rob: I think by the end of 2024 will have increased. And I think the point you made is a really good one and important one, and I think is a big part of the reason why like to your point at this point Meta is… I think their strategy of open sourcing, devoting some resources to training and releasing great open source models is generally favorably viewed and strategically pretty sound.

But if to train Llama 3 and then Llama 4 and then Llama 5, it ends up costing them billions of dollars in cash to your point, like at some point that investment, that ROI calculus will no longer make sense, as you said. And so I think that's another reason why there's just kind of relentless economic forces moving in the direction of if you invest this much into create into creating any sort of asset, including an AI model, you need to be able to have a very clear, straightforward business plan for how you monetize it.

And that business model doesn't exist for open source AI models. So I do think the provider of open source models like the Metas of the world they’re not going to keep up with OpenAI in term of the level of investment they’re making to continue pushing the field forward and I think there will be a pressure to not openly give away these models that costs so much money to invent to build but rather to charge for them basically which mean making them closed source.

So I do think OpenAI will continue to push the frontiers of this and continue to be closed source and it’s hard for me to see open source players catching up. And I know there have been some graphs around like according to some calculations the gap has been closing, but I do think it's one of these things where it's like the classic fast follower phenomenon and technology where once something is accomplished, even if the company doesn't publish a detailed, research paper about it, which OpenAI used to do, like with GPT-3, but obviously has stopped since GPT-3, but even if they don't publicly reveal all the methods around how they did it, these things tend to leak out and seep out and folks leave OpenAI and even just the demonstration that it's possible, I think, motivates others.

And so you see kind of a fast following phenomenon occur, but I think it's very different to actually leapfrog the close source models and start coming up with new methods that the leading researchers at these labs aren't doing and that piece of it, I don't see happening by the open source builders. 

Pauline: So let me push on that because one thing, you and I are both at NeurIPS right now. So hello from NeurIPS. And I think one of the big topics that I've heard Jeff Dean (Chief Scientist, Google DeepMind and Google Research) talk about, Chris Re (associate professor in the Stanford AI Lab) talk about is this idea of non-attention, beyond Transformer architectures.

And one of the big issues with the Transformer architecture is that it scales quadratically, which means that it's really expensive to scale. And one of the potential benefits of going beyond transformers is the ability to keep performance we haven't scaled that up yet, but keep performance while being able to scale sub-quadratically.

And so do you think that that's an area where open source could potentially leapfrog closed source because of these new architectures? Or are we so past the scaling laws for Transformers and it's been so optimized that it's going to be hard for any other architecture to catch up? 

Rob: It is a great question. It's a topic that I love thinking about because I think you're right that no dominant technology paradigm lasts forever. And every high performing AI model today is built using Transformers, but that's not going to be the case forever. There will be an architecture that surpasses the transformer.

 I don't know if it's going to happen immediately. I mean, there have been efforts for a year, but basically almost since the Transformer ascended in 2017 / 2018, when it was used in BERT and kind of gained this position of primacy, there have been research efforts to improve upon it. And there are so many Transformer variations and derivatives and so forth to try to more than anything, try to address this quadratic scaling issue. And none of them have really worked to this point.

There's a lot of really interesting work out of Chris Re's lab whom you mentioned that but none of it has been shown to be able to scale well the way the Transformer scales, which is more than anything the core source of their incredible capabilities. But I haven't seen anything that's directly on the horizon that looks like it's going to unseat the Transformer tomorrow or even next year in 2024. I do think it's something that's going to happen eventually, but I don't know if it'll fundamentally disrupt the open source closed source dynamics that we were talking about.

And it may, I think, this is one of the most likely vectors of disruption for a company like OpenAI, if there is some totally new architecture that entails like a totally different skill set, there's a totally different set of people who are at the frontiers of the field and all the great folks at OpenAI kind of they are incredible talent advantage lesson as a result.

But even in that world, it still feels to me most likely that the best models and the best model builders, there will be these kind of market forces that lead them to be closed as opposed to open, not that there won't be amazing open source models and not that open source isn't like an incredibly important part of the ecosystem that seeds all sorts of innovation, but just talking about the very highest performing models. It feels to me like even if there is some next generation architecture. It is most likely to remain proprietary the best performing versions of it.

And then I also want to mention just one of this kind of tangentially related to this, but a little spiel to go off on. I do think it's really fascinating to think about next generation architectures and what they could look like and along with sort of the fascination with model architecture, there is also such an obsession and fixation today on compute as a resource and the compute shortage and how difficult it is to access GPUs. Relative to those two, I think what is actually underappreciated still somehow in terms of its importance and all this is the role of data and, the quality of data that you have in the way that you curate your data and feed it into models. The dominant approach today is still like, let's take the entire internet and train on it, or let's take our organization's entire corpus of data and train on it. And the details around the data continue to be kind of an afterthought, which is crazy given that like data is the new oil is such a truism.

But I think there's too much focus on the volume on simply the volume of data, as opposed to the details of the data's composition. So I do think, relative to architectural changes, new architectural tweaks, which surely we will see in 2024, I think innovations on the data side of things are going to be so much more impactful in terms of driving improvements in model performance and also improvements in model efficiency because you can train higher performing models on less data and more compute efficiently. 

Pauline: Let's talk about that because, even at NeurIPS this week, Jeff Dean, who was really the one of the primary authors behind Gemini, which is Google's set of language models, was saying, for example that they haven't really worked on pre training models with video data. Basically the same day, there was also another researcher that was saying, yes, but at the same time, the quality of video data is not the same as a quality of text data. One trillion tokens of video data is less good or less useful than a trillion tokens of text data. And I think one thing that has been a question in the research community is when do we run out of good data to train on? And then there's a question of is synthetic data good enough? Let's set that aside for now. But do you think that in 2024, we will run out of training data?

Rob: No, another really interesting topic, this was in my predictions in that I wrote in Forbes for 2023. One of the predictions was that we were gonna start running out of training data and I think we are going down that path. I mean, we haven't run out yet. And to your question, I don't think we're gonna run out in 2024 but it is becoming a more scarce resource for sure. Because if you think about the totality I guess, starting on the language side, the totality of all the books, scientific articles, news articles, websites, etc. that have ever been produced by humans. It is a finite amounts and it's like a massive amount. But when you're talking about LLM scale datasets, it's not infinite. And so I do think we're approaching the limits of kind of the easy, the low hanging fruit, the easy-to-access language data to train on.

I think there are plenty of clever workarounds and solutions that folks are thinking about and actively working on. I mean, one big one is like the vast majority of digital data in the world is not publicly indexable. It's private data, like behind particular organizations silos. And I think it's something like only 4% of the world's data is indexable by Google. And the other 96% is private. So unlocking more and more of that, I think is a really key vector. And you see this with like a couple of months ago for instance, OpenAI announced this data partnerships program and the motivation behind it is pretty transparent, which is like, let's get as many organizations as we can to convince, let's convince them to share their data with us. And there are different ways that can be advantageous for the particular organization, but from OpenAI perspective, the benefit is just getting more high quality training data. That's that's currently locked behind these organization silos, so I think that's one dimension.

There's some interesting work around next generation OCR to basically unlock the trillions of tokens that are in like old books that haven't yet been digitized or like old science articles that aren't yet in digital format. Basically, like coming up with clever solutions to turn more of the written content from human history into a format that's friendly and digestible by LLMs. So I think that will unlock another a couple trillion tokens or whatever it is. I've heard some interesting discussion around like so much linguistic content every day is created by humans in the form of verbal communications and meetings, like this podcast right now, for instance.

Think of all the meetings are in every day across so many different organizations and most of that is not recorded. Maybe this podcast obviously being recorded probably will be transcribed, but most meetings are not recorded and transcribed and kept for posterity.

So I've heard some interesting speculation around ways to capture more of that verbal meeting data and turn that into trainable data for LLM. So I think there's a ton of like interesting strategies, but I do think like before long, we are going to start running out of the language data. And then it'll create interesting opportunities in terms of what fundamental breakthroughs are needed to kind of bring us past that barrier.

You mentioned synthetic data. I think that's one big promising avenue of research, although there's still a lot of questions about how well it will work conceptually. And then I do think some of the questions around fundamental architectures become relevant again, because if you compare AI learning to human learning, like humans are so much more efficient at learning in terms of data input compared to LLMs. The classic example that is if you show like a three year old two kangaroos, then they know how to identify a kangaroo, whereas you have to show an AI model thousands and thousands of labeled examples. So there must be more efficient ways to learn than our current methods that require like every single piece of language that any human has ever written and so I do think eventually there will be more fundamental breakthroughs that enable AI to learn just a lot more efficiently on data that's readily available.

Pauline: I think it’s definitely one of my predictions that this week that OpenAI partnered with Axel Springer, which is a global news publisher, and I think more of these partnerships are going to happen. And I think that's certainly a way that data providers can monetize in a way that is new to them. I certainly think they need it given I think there's going to be disruption and ads and potentially subscription business model. And so certainly I think that's going to be a big trend of 2024.

I want to switch gears and talk a little bit about applications because you started to mention coding and GitHub Copilot. I would argue that the most successful application of generative AI has been ChatGPT and they've crossed a billion dollars of revenue, or if you break out the API revenue, still pretty close to that and certainly, the three enterprise use cases that we hear most about are 1) AI coding, 2) customer service that has been a big area that we've been spending time on internally here and 3) content summarization. So one of the debates that's happened in gen AI is the value creation between incumbents and startups and if we just focus on those three use cases there’s so many more but let’s talk about those three as leading ones. 

So if you look into 2024 and as this war between incumbents and startups play out, what do you think we'll see in the world of applications? And certainly we can talk about the three that we’ve talk about (AI coding content summarization and customer service) but how do you think that will play out more broadly in 2024?

Rob: I think as you've mentioned, and as others I think have rightly commented, AI is a technology paradigm shift whose structure does favor incumbents more than many similarly sized technology shifts. I think that a lot of the big tech players are well positioned to capture a lot of the value that AI creates in the years ahead. And I don't think it's going to be a force that will totally disrupt a Google or an Amazon fundamentally. So I do think incumbents are well positioned, but I also think that there are plenty of opportunities for startups to emerge and win and build giant standalone next generation businesses. I think areas where startups are particularly well positioned are areas that involve net new workflows or transforming workflows in a fundamental way or even kind of net new product conceptualizations that aren't natural extensions of existing products.

And so I think like for spreadsheets for instance or making slides. Like there's an active startup ecosystem and well-funded startups going after some of those categories, reimagining the spreadsheet, reimagining kind of PowerPoint. I think I'm less bullish on those categories, honestly, I think the Microsoft’s of the world have a really strong advantage and are going to be able to feather in AI capabilities into their existing platforms, and they just obviously have massive distributional advantages.

But I think net new experiences, net new workflows that, maybe could only have been possible with AI, I think new startups will be much better position there. And I think it honestly yet to be seen, or it's still TBD if LLM powered software development fits into which of those two buckets it fits into more. I think the first iteration or the first generation of LLM fueled coding, the GitHub Copilot form factor is not that radical of a break from how software development has been done in the past.

But I do think that we're in the very early innings of seeing how AI will transform software development. And I think GitHub Copilot as it exists today will look very primitive in 5 or 10 years when we think about how LLMs have changed how software gets written and deployed. I think startups are probably in a good position to think from first principles and challenge long-held assumptions and be creative about how producing software can be fundamentally transformed using this new set of tools we have at our disposal.

And so I think software development may as one of the few categories that you mentioned is from the center in terms of killer applications for LLMs. I think software development may prove to be one of these categories where startups with like a really disruptive kind of blank sheet vision of, of what coding can look like in the future may be able to actually emerge and be successful.

Pauline: Can you give an example of what you think maybe one of the most exciting new workflows are that you think could be really interesting for startups to go after?

Rob: I think one example or one category of examples is tools that can automate really meaningful swaths of knowledge work in different categories. It's kind of become a little cliché that like Copilot for X, Copilot for Y, but I think platforms that can automate enormous chunks of what lawyers do today or automated enormous chunks of what tax experts do today or even data scientists. I think those kinds of tools will in the not too distant future will eat giant chunks of what lawyers or tax attorneys or data scientists do, and as a result, free up those highly educated knowledge workers to focus on the more creative and important high value aspects of their jobs.

And so I do think in at least some of those cases, there will be an opportunity for the human to hand off a lot of that fairly grunts formulate work, which today accounts for most of how they spend their hours and devote themselves in very different ways to kind of the less, formula like parts of the job. And so I think in some of those categories, I can envision really different transformative applications being built that don't have a clear analog in today's software stack and that aren't easy to just layer on an existing software product, but that will become very central to how those occupations carry out their work.

Pauline: I'd say the one that I always go to is the one that Character AI is going after, which is AI friend. I don't think any of us even two years ago would have said, you can hold a two hour conversation with an AI chatbot and, and yet it's happening. I would say I broadly very much agree with your framework.

I'd say the only other comment that I would make is I think when people talk about incumbents, people think about Microsoft and Google and Amazon, and those are formidable competitors for sure. But one area of incumbents that we've been thinking about are more legacy players that have very large operational footprints, whether it's in customer support. And take PayPal, for example. I think they've spent something like $2 billion a year in customer support and operations or about 8% of net revenue. And so if they can even cut that by 50% and get a 4% margin increase, that's meaningful for companies that are trading on a net income or earnings basis. And so that's one area that incumbents that I say, we're thinking a lot more about.

I know that you spoke about GPUs and the bottleneck at Ted AI and I really enjoyed the talk. I'm curious, one prediction that we have going into 2024 is that the bottleneck shifts from actually having the physical GPUs to actually how do you install them in data centers and allow them to be used efficiently, and so the bottleneck is more of the power transformers, the cooling devices, the physical real estate that is needed to house these racks of GPUs. I'm curious, do you think that 2024, the GPU rush will abate? And by how much, or do you think that this is just a trend that will continue into next year?

Rob: I think the GPU shortage is very much a moment in time thing. Like it feels all consuming and the AI world is really fixated on it. For good reason, any company that is building big models, like this is their constraint today in 2023, being able to access GPUs, but it definitely will pass. I think it's like any supply and demand dynamic. A ton more supply is coming online to service the demand. There's obviously, there's a longer lead times and for many products, just given the complicated supply chain and so forth.

But I do think the GPU shortage certainly will ease in 2024 and what will it be completely adressed in 2024 or 2025, like, exact timelines are hard to predict. But definitely, I think that this chip scarcity will abate, and I think it'll be a really good thing for the AI industry overall, because it is a shame that currently, like, there's more innovation that could be happening. There's quicker execution and quicker shipping that could be happening if companies could just access the hardware that they needed to. I think it'll be a good thing in terms of, like, opening up the gates of more activity, more innovation.

I also think that the GPU shortage and I guess, in addition to NVIDIA ramping up its production, there are alternatives that are coming online. And it's just inevitable that NVIDIA has like dominant grip on the market, it’s not going to last forever as amazing of a company as it is. So, AMD recently came out with their GPUs, which early reviews that I've heard are promising in terms of the chips capabilities. Intel has come out with chips that I think people are tend to totally discount Intel as a player in the AI accelerator market, but I think they're going to be coming out with chips that are increasingly competitive.

And then on the inference side of things. I think inference is going to get more and more efficient, and it's going to become more and more viable to do inference on a wide range of commoditized hardware, including CPUs. So I definitely think the insane bottleneck that we're all obsessed with right now is going to ease. And I think it will lead to, I think there's been some kind of funny, unnatural behavior in the market this year as a result of the shortage.

There are companies who basically function as GPU resellers and there are different narratives that they have in different ways they position themselves, but there's a whole cohort of startups who have seen incredible top line revenue growth this year because they basically basically everyone will pay a lot of money to get access to GPUs these companies are reselling them. And I think that those that revenue growth is going to slow next year. And not all of those companies are going to prove to have something really valuable that they built on top of the GPU access that makes them durable businesses. So I think that category is going to come back down to earth.

I think the insane revenue growth that some of the upstart cloud providers like Coreweave has seen is not the revenue growth won't be sustainable. That doesn't mean Coreweave won't figure it out and become a great business. But I do think a lot of these dynamics were the product of this somewhat artificial scarcity and thing. It does feel inevitable to me that things kind of realign in the next 12 to 18 months. 

Pauline: I certainly do think for those GPU players, it's a race to actually build software and value add on top of it. And they've struck this amazing gold mine. And now I think that the clock is ticking in terms of building other things on top of GPUs. And so I would agree with that.

I probably have a different view on the GPUs, which is I think 2024 is probably not going to be the year that we see abatement, but mostly because I think just people are still in so much of discovery mode. And I think the big labs, to our earlier conversation, still needs so many chips to train these next generations of models that I think next year will still be in a bit of a bind. But I think in 2025, it'll definitely loosen up.

And I think, the other component is I think there's a lot of work that's being done into how in terms of how to make the GPUs that we currently do have more efficient and sort of whether that's when you're not using it, how do you allow other customers or other companies to access it or other unique ways that we can make them more efficient? So I think that'll also help going into 2025. 

Well, with that, let's go into Rapid Fire. First Rapid Fire question: what is your definition of AGI? And when do you think we'll get it?

Rob: That is a good, hard hitting Rapid Fire question to start with. I think AGI is an overloaded term that there isn't a good definition for. I actually wrote a whole article on Forbes last year about how the concept of AGI  is oversimplified to the point of meaninglessness, and it just means a lot of different things. But I do think the discussion around I think there are real, there are fundamental problems with the term AGI. But I think the discussion around runaway superintelligence and when it might happen is still important to have, because there's doomers on one side and there's acceleration on the other side.

It's almost become this, like, polarized, almost like politicized debate. In some ways, it reminds me of the of the Don't Look Up movie. It's just like a caricature of what politics looks like in the U.S. today is sadly what the discourse I feel like is kind of starting to become. But I mean, I think no one knows for sure what the growth curve looks like for capabilities from here on out. And it's possible it will plateau. It's possible it will continue to exponentially grow, but there's at least a scenario in which AI does become far more capable than humans in the not-so-distant future, like, in the next few years, I wouldn't necessarily say it's likely, but it's certainly non-zero.

And so I think people that are more on the worried side and are taking that as a more serious possibility. I think it's a very good thing that there are people that are diving in on and thinking through the implications of it, raising the alarm about it. I think it's healthy and important to have that line of discourse. And it may not come to pass, but it may, and so I think it's a good thing that we're thinking about and talking about super intelligence, however, we want to frame it.

And I think, dismissing that line of thinking altogether or being dismissive of or labeling people who are worried about runaway capabilities as tumors and technology pessimists and so forth I think totally misses the point and eliminates a lot of nuance in the discussion and kind of falls into this reductive ad hominem attack. So I think it is a really important discussion. I'm glad that folks are having it. I think nobody knows for sure. The only way that you can be positive that someone is wrong on this topic is if they're 100% sure that their rights because really nobody knows. But I think it's a good thing that we're talking about it and that we're worrying about 

Pauline: Absolutely. Second Rapid Fire question: what's your mental framework on regulation?

Rob: It's an important topic. This year, obviously it was a year that the rubber hit the road in terms of regulators getting serious about AI. Europe has passed the AI Act. As usual, Europe is ahead of faster than other geographies in terms of passing regulation, just as we are. But in the U.S., regulators and lawmakers are obviously starting to get a lot more serious about it. Folks in Congress are talking about at the White House is very fixated on it. I think it'll probably be another couple of years at least until Congress actually passes a piece of comprehensive legislation around AI.

U.S. Congress takes a while to get things done, but I think it'll happen eventually. And the White House obviously is doing everything it can using its executive powers. Tt remains to be seen how much that will matter, if at all, if a different parties in the White House next year. Then, the first thing they'll do on day 1 is just get rid of it and it won't have any impact. And even if Biden does have a 2nd term, there wasn't anything in there that was that transformative or disruptive, but certainly it's something that regulators are thinking about very actively now.

I write a column in Forbes about big picture themes in AI, and every year I write a column with 10 predictions for the world of AI for the following year. So I'm just now working on my 10 predictions for 2024, which I'm gonna publish next week. But a few of them, two of them relate to regulations, so I'll give a sneak peek and mention them.

Pauline: Would love that.

Rob: And then, when we can see this time next year we can do another podcast, we can see whether or not these actually came to pass or not, but. So these are more specific regulatory issues that I faces that are kind of live questions.

The first was around IP and copyright. And the issue that a lot of people are aware of is foundation models that are trained on the entire internet, which is most well known foundation models today are trained on a ton of data that the model builders don't have legal rights to - either language that was scraped from books and articles and so forth, or images that were created by artists.

So there's a really big looming question of copyright and is OpenAI, is Stable Diffusion, is Midjourney, are they all violating copyright protections by using this data as training data for the model? And it all hinges on this question of fair use, the doctrine of fair use and copyright. And is model builders use of this AI? Does it fall under the fair use doctrine? First of all, it's going to take years for this question to be fully worked out in U.S. courts.

I feel confident it will eventually go to the U.S. Supreme Court. And that's how it will get resolved. I think it's an important enough issue will go to the Supreme Court. It won't work its way through the courts all next year. It's going to take longer than that. But I think that next year, at least 1 court in the U.S. 1 lower court will rule that model builders use of Internet data is a violation of copyright. All right. And that's going to send a lot of people in spirals panicking and freaking out about the business implications for model building companies.

That won't be the final word again. As everyone knows, if you lose a case in lower court, you can appeal it. And you can appeal it again, and eventually it goes to the U.S. Supreme Court, but I do think there will be some, I think some case law will come down on that next year that will be, less favorable to model builders, and I think that'll be an interesting dynamic to keep an eye on this.

Pauline: probably won't be next year, but do you think this will ultimately go to the Supreme Court? 

Rob: I do.

Pauline: And you mentioned one more prediction that you had, that was relevant to regulation in AI?

Rob: Another kind of regulation related one is around, as folks are aware, a lot of the big cloud providers have been making large investments into startups that include those companies turning around and spending a lot of the money that they raised from the cloud on compute that the cloud provides. You often hear this practice referred to as round tripping. Because, the. Money's going out the door in the form of the cash is going out the door in the form of investment. But then coming back to the cloud companies, I think this practice will come under a lot of regulatory scrutiny.

Next year, and it just intuitively it's seeing it's like a little bit too good to be true of  a tactic from an accounting perspective. And it basically is a method that lets a cloud provider turn cash on its balance sheet into revenue in like a zero risk way. And so it doesn't, it feels like, I mean it seems esoteric maybe in terms of accounting practices, but it has significant implications for how these companies get funded.

 I expect regulators will like come down on that practice and if not put a stop to it altogether, then at least like heavily limit the extent to which it happens. So I think that's another. Interesting regulatory dynamic to keep an eye on. 

Pauline: I was actually thinking that there may be more, I totally agree with you and I was actually thinking, like, does next year bring actually a full of the big models because the relationships between these independent. So called independent. Cutting edge model makers are so complex with the CSPs, whether it's Microsoft or OpenAI, where I think Europe has sort of, it's trying to figure out if they want to bring a case, to the courts and stop it with both actually Amazon and Google.

And so. I think the corporate ventures have been very, very active in AI this year. And I'm very curious what that means for next year as the numbers just get larger. And all of this, you could argue is just a rounding error on their balance sheet. That may not be the case next year if these, the value of these, investments continue to go up and the dollars continue to go up.

Second to last rapid fire question, who are one or two of the biggest influences on your mental framework on AI?

Rob: I'll try to keep this one to actually Rapid Fire. I know that last one wasn't very rapid. So I really like going back and reading thinkers on AI from earlier, eras earlier, decades, because the fact is the technologies and methods that are dominant today are new and are constantly being innovated on.

But a lot of the more fundamental questions, conceptual issues around what would it mean for a machine to have intelligence? What is intelligence? What implications does that have for humanity? These are questions that folks have been grappling with since the, since AI became a real academic discipline in the 1950s.

And I think a lot of the most powerful writing on this was done years or decades ago. So Douglas Hofstadter is one author who I really like. He wrote a book called Gertle, Escher, Bach, which is kind of a famous iconic classic book in AI he wrote it in the 1970s. But it's still, I think, very salient in terms of thinking about AI and implications of AI and how I might come about.

Marvin Minsky wrote a lot and I mean, a lot like Marvin Minsky, for instance, I think a lot of his technical views proved to be very wrong. He was a longtime skeptic of neural networks and didn't think they would work. But still, I think a lot of the conceptual ways he thought about AI are fascinating. He wrote a book called The Society of Mind that I think has a lot of great insights into it.

Even Alan Turing, I mean, I don't think Turing wrote any books that I'm aware of, but a couple of the papers that he wrote, like the 1950 paper where he starts out with I want to examine the question, can machines think, and he introduces the concept of the Turing test, which he calls the imitation game. I think that paper has a ton of fascinating insights. And he was like many, many decades ahead of his time in terms of how he was thinking about machines. So anyway, I think there's a lot to be learned from some of these older school writers who were writing before the current deep learning era, but thinking about a lot of the same issues.

Pauline: I love it. I'll have to link those in in the show notes.

And last Rapid Fire question. My normal rapid rapid fire question is what is one thing that you believe strongly about the world of AI that you think most people would disagree with you on in light of this episode? Would love to hear what your hottest take for 2024 is in terms of a prediction.

Rob: Always fun to talk about hot take.

Here's one from me, that may be controversial among the AI crowd. It almost became like a common theme or trope this year that like, the crypto to AI pipeline. And there are all these VCs especially, but also founders and commentators and others who were super excited about crypto and all kind of aggressively rebranded themselves as AI people this year as AI got hot.

I'm going to predict that in 2024, some of that like herd mentality and like trend follower behavior is actually going to shift from AI back to crypto. As I'm sure a lot of people have been tracking, Bitcoin is really starting to rip. Crypto is just unbelievably cyclical industry, and it has crazy ups and crazy downs but it will come back up.

I know it's like out of fashion right now for the most part, but it will be back and it will be buzzy again. And I don't know for sure that 2024 is the year, maybe 2025, but it feels like 2024, maybe the year that crypto becomes buzzy again. And I do think that, right now it's hard to imagine VCs getting excited about anything other than AI, but a year is a long time and for better or worse, I think a lot of VCs, their “convictions” can be incredibly short lived. And we saw a lot of that with them pouring into AI over the past year, and I can see a lot of them moving on. I'll admit, some of this is probably hopeful thinking on my part, because I personally would like it if there was a little bit less fuss and noise and overexuberance around AI. But that's my hot take that the crypto will come back into fashion and thankfully, hopefully some of the crazy hype around AI will be diverted in that direction. 

Pauline: I love it. That was actually not a take I was expecting. I agree with you. I do think, I mean, we've already started to see some of the VCs start to announce, very buzzy crypto funding rounds. And so that might be a very good signal that your 2024 prediction is going to come true. 

Rob, thank you so much for spending the time. Really enjoyed the  conversation. I know we covered a lot of different topics. I think if 2023 is any signal as to how AI is going to unfold, I think 2024 is going to be insane. And so I really appreciate you taking the time to come on and speak with me. 

Rob: It was great to chat. Thanks for having me.