AI Is Transforming Businesses (with Andrew Ng)

AZEEM AZHAR: Hi, I’m Azeem Azhar, founder of Exponential View and your host on the Exponential View podcast. When ChatGPT launched back in November 2022, it became the fastest growing consumer product ever, and it catapulted artificial intelligence to the top of business priorities. It’s a vivid reminder of the transformative potential of the technology. And like many of you, I’ve woven generative AI into the fabric of my daily work. It’s indispensable for my research and analysis. I know there’s a sense of urgency out there. In my conversations with industry leaders, the common thread is that urgency. How do they bring clarity to this fast moving noisy arena? What is real and what isn’t? What, in short, matters? If you follow my newsletter, Exponential View, you’ll know that we’ve done a lot of work in the past year equipping our members to understand the strengths and limitations of this technology and how it might progress. We’ve helped them understand how they can apply it to their careers, and to their teams, and what it means for their organizations. That’s what we’re going to do here on this podcast. Once a week, I’ll bring you a conversation from the frontiers of AI to help you cut through that noise. We record each conversation in depth for 60 to 90 minutes, but you’ll hear the most vital parts distilled for clarity and impact on this podcast. If you want to listen to the full, unedited conversations as soon as they’re available, head to exponentialview.co. I’m really excited for our next conversation. I’ll be talking to someone who’s been at the forefront of not just AI research but also of education and the commercialization of these technologies for several years, an accomplished researcher who built AI organizations of both Google and Baidu. He also founded the world’s most important massive open online course provider, Coursera. And now, for the past several years, he’s been helping companies come to grips with deep learning. Andrew Ng, it’s wonderful to have you back on the show.

ANDREW NG: Thanks, Azeem. I’ve been a fan of your work for many years. It’s great to be back.

AZEEM AZHAR: We last spoke four years ago, almost to the day, and a few weeks before that, we’d met in person in London in 2019. And, of course, so much has happened since then; of course COVID, but also, the development, maturation, perhaps even the breakthroughs, in AI and in deep learning. I mean, if you reflect on those four years, what are your key takeaways?

ANDREW NG: About 10 years ago, deep learning, which is good at labeling things, started to work really well, and I think we’ve been all figuring out use cases for deep learning as a general purpose technology. And then, of course, about a year ago, ChatGPT and generative AI really… A little bit more than a year ago, even before ChatGPT, generative AI started to take off with images and texts and as general purpose technologies, meaning technologies that are useful not just for one thing, but for a lot of different things. It feels like a lot of us in AI are even busier than ever because the number of applications that AI can now tackle is bigger than ever before.

AZEEM AZHAR: Yeah, it is remarkable. I think people struggle to get to grips with the idea of the general purpose technology. You can use it in sales, you can use it in HR, you can use it in finance, in product development, in coding; but, you can also use it in every single industry, and you can construct new industries from it. It almost becomes hard to make it a tractable business opportunity because there are so many places this thing can be applied.

ANDREW NG: Yeah, it is one of the difficult things to talk about or to think about in terms of AI as a general purpose technology. It’s kind of like if I ask someone, “What is electricity good for?” It’s almost hard to answer, because it’s useful for so many different things, and AI is like that, too. I think in history of humanity, the internet, electricity, we’ve had a few general purpose technologies, and those technologies tended to bring forward major leaps in innovation, create lots of businesses. And, one of the things I’ve been excited about is how can businesses take a systematic approach and ask 10,000, or 100,000 employees, or however many, whatever large size a firm is, how can you figure out how this will affect your business, and what are the new opportunities? So, going about that systematically has been one of the most exciting emerging frameworks and methodologies.

AZEEM AZHAR: Right. So, is that something that you’ve been doing, then, when you’ve been working with businesses, which is to try to get some bottom-up input in how people on the frontline think they could apply these techniques?

ANDREW NG:  Yeah, so there’s actually a systematic process of… It turns out AI automates tasks rather than jobs. I know people think about AI automating jobs, but I think, from a business perspective, a more useful way to think about it is jobs are bundles of tasks. So, take a radiologist. A radiologist resets their images, they have to gather patient histories, they have to consult with other doctors, they have to maintain, operate the machines. So, radiologists, and really most jobs, have a lot of different tasks.

AZEEM AZHAR: And this is why, by the way, we still have radiologists. I remember that Jeff Hinton, I think six years ago at an event in Toronto at one of these disruption events, said, “We won’t need radiologists,” and, “Don’t become radiologists.” And Jeff Hinton, of course, is an absolute esteemed computer scientist, but what I think he missed out when he made that comment was your point, which is that the bit that a computer currently automates within radiology, which might be the markup of a MRI scan or something, is only a small task in the full task lifecycle of the radiologist, which is why six years later, with vastly improved AI systems, we have more radiologists than ever, and the shortage of radiologists in the US and in Europe is bigger than it has been as well.

ANDREW NG: Yeah, I think that’s exactly right. And, in fact, one of my friends, Kurt Langlois of Stanford, made a counterpoint to what Jeff Hinton said, that I agree with, which is, “Radiologists that use AI will replace radiologists that don’t, but it’s not that AI will replace radiologists.” And, when I work with businesses, I often find that, when we understand what are the tasks the employees are doing and look at where are the automation opportunities, there are often some. There are almost always some. But there’s often, I don’t know, 20% of the tasks, and this means that most people’s jobs are actually quite safe. But also, the productivity boost can be very significant. I think, depending on what report you trust, I’ve seen numbers on the low end, maybe 15% of tasks done in the US can be automated. On the higher end, I’ve seen estimates going up to close to 50%. It is a huge range, but that is a lot of economic value that remains to be identified and then executed on.

AZEEM AZHAR: Yeah, and I think that there’s a really interesting dynamic to this, because, when you think about the radiologist, the radiologist has all sorts of power, in the sense that the job is prestige job. It is a job that is embedded within a hospital system. You’re at the top end of that hospital system because you’re a clinician. You’re not a cleaner. And so, when it comes to the way in which the technology makes its way into the organization, you have a certain amount of say in terms of how protocols change and how it rolls out. But, I think there is the other end of this, where there are a lot of people who are doing jobs where key tasks, maybe key subsets of tasks, could be automated, but they don’t have the same level of power in the organization, or the job is a lot more fungible. They can be swapped in and out for someone else. And I’m thinking about, say, first line customer service. Not tier three, but the first line customer service, where a large portion of the queries that you might receive may be very, very standardized. And, tenure is not so long in those jobs. There’s a revolving door of people coming in and out. When I look at applying generative AI in that market, I think the power dynamics, the skill dynamics, and the relative perceived importance of that business unit to the CFO or the CEO might say that how that plays out, in terms of jobs, would be different to something super high-skilled like radiology or oncology. I mean, is that your sense? I mean, you are actually working with businesses, so you have data rather than theory you can throw at this.

ANDREW NG: I partially agree with you. In fact, some of my friends, Eric Brynhofsson, Andrew McAfee, and Danny Rocks, some of the collaborators, are working with a lot of businesses on these types of issues. I chat to them, get the input as well, and my sense is that, while power dynamics can accelerate or slow down adoption in some businesses, I think is a factor than just the sheer economic assessment of what are the tasks, how valuable are they, as well as the technical assess factors. There are a lot of things that radiologists do that I don’t know and I think no one knows how to get an AI to fully automate right now. So, no matter the power dynamics, we just cannot replace radiologists today. Whereas, in contrast, with contact centers, which AI is having a massive disruption on, lots of tasks are being done, they just factually can be done, just as well, sometimes better, with a AI chatbot.

AZEEM AZHAR: Right. And, you said it’s having a massive impact today, so do you have some examples of the kind of rollout of these tools in, say, a contact center, and what the before and after looked like?

ANDREW NG: Oh, so… Boy, in private conversations, I probably shouldn’t name any companies, but I’m definitely hearing many, many stories of contact centers being disrupted. We have people saying, “Yep, we just saved 30% of our costs in a contact center,” because they’ve been able to do automation. And in fact, even years ago, before the rise of gen AI, I was involved in building a contact center automation thing. We found out that about 10% of the customer service requests were asking for a refund. We just did deep learning to recognize that, then copy pasted the form to request, and then that diverted about 10% of the calls. So, I think this is actually… And it definitely gained steam. This movement definitely gained steam in the last year with large language models and generative AI.

AZEEM AZHAR: But what you’ve described, I think, is a really healthy process that goes on in any organization anyway, which is that you do look to automate repetitive tasks, because humans are not very good at doing repetitive tasks without making mistakes anyway. So, that’s why, when you look at a repetitive task like airport security, you have to have all these processes around it to ensure that the needle in the haystack still gets spotted, that people don’t switch their attention. I think that organizations who don’t try to enter into a kind of process of, like Kaizen, right, continuous learning, continuous improvement, will always fall behind. So yes, it’s not surprising. I mean, I remember working with a couple of contact centers, even pre-deep learning, and you would do this analysis. The tools were much, much worse. And, you’d come up with really simple scripts, perhaps in Python, to triage what you’re doing, to pull things away from going to the front line human because it was just going to be more error-prone and much more expensive. It’s not a conspiracy, in a sense, it’s just a business continuing to learn, continuing to innovate, continuing to improve. But, then I suppose, you get this moment of a step change in the technology technical capability. I’m curious about whether you, just thinking back to when we last spoke in 2019, whether you think there has been a step change in the capabilities of AI for working in the fuzzy world of human text and human speech? And, it really is like a step change, a paradigm shift, or an inflection point, where it was really a case of accelerated linear progress.

ANDREW NG: I feel like what we can do now with generative AI is definitely vastly bigger than what we could do a year ago. I’m always nervous to talk about step changes, because I think a lot of these, things they are maybe consistent with the name of your pod, with what you do, and exponential change. But, often, exponential changes look a lot like step changes. And then, to be technical, it still remains to be seen if it will turn out to be a sigmoid and will plateau at some point. But, right now, it still feels like we’re on the up and up without slowing down yet.And, just to share one lesson, since I know a lot of your listeners have businesses. One lesson I’ve learned doing this myself with quite a few large businesses is that the task most amenable to AI augmentation or automation, they’re often not what you would expect at first blush. So, it turns out the radiologist example we’re chatting about, we have this image of our heads of radiologists reading X-rays. It turns out, that’s actually pretty hard to automate, but there are other things, like gathering the patient history, maybe other things that they actually do, that could be easier. Or, take computer programming. If you picture a computer programmer, you think, “Well, computer programmers write code. That’s the iconic defining task of computer programmer.” But, if you actually analyze all the tasks that computer programmer does, they also debug codes, write documentation, and I think that, in that type of analysis, we often find that they’re easier things to automate than that iconic defining task that we tend to think of when we think of a job role. So, I find that, my team AI found, we work with quite a few large businesses, and in our portfolio company work, Helix works with quite a few companies to do this analysis. We do this because it identifies opportunities for AI people to go and do meaningful work, but often, the outcome is not what one would guess at first blush.

AZEEM AZHAR: Right. So, your inbound assumption is, “Oh, we’re dealing with a coder, a developer, and what we should do is help them write better code.” But it may be that a more effective mechanism is to have the AI system debug the code or to write the documentation or to review the product requirement documentation that the product managers have produced, right? It’s the ancillary tasks rather than the final actual bit of writing where the benefit may come.

ANDREW NG: Yeah. Yep. And it turns out, I think AI is actually pretty good at writing documentation already, whereas writing code, it’s okay. It’s getting better, but it’s still needs a lot. Not that great yet. So, I think these analysis… Or, another example, we’re working with aN agriculture company, where there’s an iconic image of people doing certain things, but found out, I believe, that it’s a transportation of the harvested things that had to create this opportunity. So, you see a lot of different surprising outcomes like this.

AZEEM AZHAR: Yeah, and I suppose that’s where, again, the perspective of the technologists building the technology who are not domain experts in agriculture or contact centers or regulatory compliance is always a little bit constrained, because their expertise is actually in getting these systems, the AI systems, to work well, rather than their specific application. But, I wanted to come back to this question about the last four years. I would say that a kind of common media understanding, perhaps the way in which things have been presented by the big tech firms and some of the AI startups, is that there has been an acceleration in the rate of progress in the domain. I mean, what’s the best way, do you think, to measure the rate of progress? And, I’m doing, for podcast listeners, I’m doing air quotes around rate of progress. Right? “The rate of progress.” What’s the best way to measure that?

ANDREW NG: Gosh, I wish I knew. The AI field has tons of benchmarks, and I guess, in terms of benchmarks such as how accurately can we route email or solve some logical puzzle or whatever, probably it is true that, on benchmarks like that, the rate priorities has accelerated. And then, I think that, with large language models, like ChatGPT and then BingChat and Bard, I think the consumerization of AI, where a lot more people use it day-to-day, that’s certainly accelerated. I don’t know. I think benchmarking AI is a whole giant field to itself that people still debate about. It does feel like things are moving faster and faster, though. I feel like the sheer number of people using it directly, as well as the amount of capital being invested, those are very large forces that leads to even more innovation and acceleration as well. So, I think it’s clearly accelerated.

AZEEM AZHAR: Yeah, no, I think that’s right. I mean, more people rather than fewer looking at a problem, you would think logically, should speed up our ability to tackle the problem, especially if there is any underlying competition, and especially as if there is some heterodox thinking. So, even though people say, “Well, everyone’s focusing on large language models,” there are a lot of different innovations that are going on within them. So, you have, on the one hand, OpenAI, which has had, for a long time, the belief that scale is going to be absolutely critical, and that will come through the amount of data and the amount of compute. And then, you have these very reliable scaling laws that tell you what performance might look like for given data and given compute. They will pursue one direction. And then, you have this whole series of other companies and projects who are saying, “Well, can we be more parsimonious with our approaches?” And I think, one of the things that I found interesting, is that we’re a year on, actually a year on and a couple of days when we record this, from ChatGPT being released casually into the world, and there is such a rich selection of large language models that is now available. There’s a much richer one that’s available now than there was a year ago, and probably, if I had asked myself the question, “How rich will the environment be in a year?” I think I would’ve underestimated where we are today. I just thinking about not just what Meta has done, but all of the forks off its llama open source system, suggests to me that there’s the kind of diversity that gives rise to the dynamics of greater innovation.

ANDREW NG: It’s been interesting, how long scaling has gone on. When I started Google Brain, a decade-plus ago, the number one directive I gave the team was, “Let’s build really, really large neural networks and train them with a lot of data.” And, that scaling recipe drove Google Brain and then many other teams forwards. And, I remember, I think it was 2017, when I was leading Baidu research, my team at Baidu Research actually published something called Deep Learning Scaling is Predictable, Empirically. So, if you Google online, you can still find Baidu had one of the earliest, I don’t know if it was the earliest, but one of the earliest research articles on scaling. I think we… One thing we missed was, my former team did it on LSTMs, because transformers weren’t quite a thing, that, but I think that actually even LSTMs or earlier technical architecture. Similar scaling roles.

AZEEM AZHAR: We had here Jürgen Schmidhuber, who was involved in LSTMs as a guest a few years ago as well. Yeah.

ANDREW NG: Yeah. So, I think this concept of scaling continues to be working today, which is exciting.

AZEEM AZHAR: I mean, it’s exciting because it gives us something that is reasonably predictable, and that helps in terms of unleashing investments, because you can say to your funder, “If we do five x in dollars in for compute and data, we expect to get this kind of return.” And someone can figure out whether that’s going to be economic, right? It turns it away from being completely open-ended research into something that’s a bit more… I guess, a bit less risky.

ANDREW NG: Yep. And then, the scaling loss has definitely been a convincing case to many investors. But, in terms of investment, there’s one sector that I think tends to be under looked, which is the application layer, because we tend to read about the massive amounts of capital and all the exciting work done by the tool builders, by the transformer or the foundation model trainers. But, it turns out, that for all the tool builders to be successful… I think, large language models, they’re a great consumer tool, but they’re also a great developer tool. And, they’re letting people write other software on top of them, for contact center automation, and parsing of legal documents, and writing marketing things, and so on and so on. And, it turns out, that in every wave of technical innovation we’ve seen, for the tool builders to be successful, the application builders have to be even more successful because we kind of need them to generate enough revenue, even more revenue, so that they can afford to pay the tool builders. I feel like the media tends to talk about the tools in the tech, but I think that there’s an even greater set of opportunities. In fact, there has to be, mathematically, for this whole ecosystem to be successful at the applications.

AZEEM AZHAR: Yeah. That’s actually a really great point, right? Because, if you don’t have the end revenue from consumers or businesses who are doing something useful with it, you can’t take a portion of that revenue to pay the tool builder who then has to pay the foundation model company. And right now, that gap is being made up by equity capital from investors. But, that doesn’t become sustainable, and I guess that’s what happened, in a way, in cryptocurrency, was that there were no real consumer applications. And, even outside of the speculation, all you had were tools and foundational networks, layer one blockchains, but no consumer utility at the end. At some point, the merry-go-round runs out of steam, right, it runs out of momentum. In the case of crypto, they maintained the momentum through the hype and the pump and the dump. So yes, for a healthy ecosystem, yeah, you’re right, you need to have applications that are out there that are being used quite heavily. I think sometimes about this. I think, where are the real consumer applications of generative AI? And it’s not been long, Andrew. As you say, it’s a year since ChatGPT, and maybe a year and a half when the image models from people like StableDiffusion and Dali and so on were starting to show real potential. I can point to perplexity, which is a kind of search engine alternative, and I can point to ChatGPT and Adobe’s generative AI product, but I guess I can’t point to many gen AI consumer products that I think are heading into the tens or hundreds of millions of dollars of revenue a year right now. I’d be curious if you know of some.

ANDREW NG: I think that I’m seeing a lot of… So, ChatGPT by OpenAI, that is a consumer application that is generating meaningful revenue. And then, I think there’ll also be a lot of P2P use cases. So, I’m seeing exciting work done on gen AI used for dealing with legal documents or dealing with healthcare settings. I think the contact center automation, that’s a significant sector. And, this is why finding all the use cases of gen AI is going to take longer than any of us wish, because someone needs to go figure out, “All right, in a healthcare system, where exactly should we use gen AI?” And then go and build it and test it, maybe get regulatory approval if needed, and then, to not just do this in healthcare, but in financial services and education, and on and on and on. Oh, you can have copilot. I think that reported over a hundred million in ARR as well.

AZEEM AZHAR: That’s right. Yeah, yeah.

ANDREW NG: I think it’s nice to see a few. Certainly a few. So, I’m actually optimistic we will get there collectively as a community. But then, I’m also thinking, boy, we better get there as a community, because otherwise, this will collapse. But I’m optimistic, because it does have a lot of use cases.

AZEEM AZHAR: But we may also just be a little bit impatient, because, right now, it has only been… The transformer paper was 2017, right? So this is the first scientific paper, non-peer reviewed, that talked about this architecture that we used. To get to a stage within six years of there being multiple hundred million dollars consumer end user products is quite fast compared to, I think, many other technology waves. One of the things that struck me about gen AI and holding these different ideas in your head was a survey from… Well, OpenAI themselves said that 92% of the Fortune 500 have developers using their API. Now, this could be a single developer, this could be 50 people. But then, Tom Malone at MIT did a survey of chief digital officers, which said that 19% of them had active department level projects on gen AI. And, there are two things I took away from that. One was that 19% in a year is extremely high for a new technology for department level experimentation. You didn’t see that in cloud or in mobile in the first year. But the second thing is, the fact that 92% of developers are already playing around with the tool told me that the point at which those companies, who weren’t experimented, started to experiment. They’d do so in a space with developers who’d already built their own little mini applications which could really speed up those projects, within the context that projects always take a long time in a large company. But, I was seeing indicators that uptake is both, and could be, faster than the previous major tech waves, which, in this case, I think would be mobile and cloud. Well, thanks for listening. What you heard was an excerpt of a much longer conversation. To hear the rest of it, go to exponentialview.co. Members of Exponential View and the community get access to the full recording as soon as it is available, and they’re invited to continue the conversation with me and other experts. I do hope you join us. In the meantime, you can follow me on LinkedIn Threads and SubStack for daily updates. Just search for Azeem. A-Z-E-E-M. Or, if you’re in the US and Canada, A-Z-E-E-M. Thanks.