Azeem’s Picks: The Promise of AI with Fei-Fei Li

AZEEM AZHAR: Hi, there. It’s Azeem Azhar, our founder of Exponential View. We are moving into an age of artificial intelligence. These tools of productivity, efficiency, and creativity are coming on in leaps and bounds even if they remain incomplete and immature today. Implementations of AI are becoming priorities amongst top execs in the largest firms all over the world. Now, one big question is how do you make sure your AI systems behave ethically and fairly? It’s a huge issue and it’s one I’ve been exploring since 2015 in my newsletter, Exponential View. And over the years, I’ve hosted some of the leading experts on this subject, on this very podcast. I know that ethical AI implementation is top of mind for leaders like you. So, to help you think through the questions of responsibility, accountability, and power in the context of AI development, I’m bringing back some of my previous conversations over the next five weeks. Now, this week I bring back my 2020 conversation with Fei-Fei Li, a pioneering computer scientist whose work transformed the field of machine vision and kicked off much of the current AI frenzy we’re witnessing today. Fei-Fei is a professor of computer science at Stanford University and is the founding Director of Stanford’s Human-Centered AI Institute. She explores her two decades of work at the forefront of AI in her new book, The Worlds I See: Curiosity, Exploration, and Discovery at the Dawn of AI. I’m certainly picking up a copy, but in the meantime, here’s my 2022 discussion with Fei-Fei. Fei-Fei, it’s wonderful to have you with us today. Thank you for taking the time.

FEI-FEI LI: Thank you, Azeem. I’m very excited and really have been a fan of your program.

AZEEM AZHAR: Today, I’m in London, we can’t travel because of the COVID lockdowns, where are you today?

FEI-FEI LI: I’m at Stanford, so we’re all locked down. Zoom and Slack is where I live on. One thing I love about my job is just hanging out with the young people and being challenged and refreshed by them every day. So that aspect, I really, really miss.

AZEEM AZHAR: The talent within your research lab that is aligned around robotics and machine learning and artificial intelligence, some of those skills seem so relevant to tackling issues that are being raised by both the disease itself, COVID-19, and some of the impacts it’s having. How has your lab lent into this?

FEI-FEI LI: Eight years ago, and we were not predicting COVID, we realized computer vision and smart sensors and devices have moved into a stage where we can start tackling some of the real-world problems, especially my passion and real world problem has been healthcare. So we’ve been piloting some research on contactless sensors and trying to understand human behavior that’s relevant to clinical outcome. And one of the major area that caught our attention is the issue of the world’s aging population and how we can help elderly to live more independently, more healthily, yet well-connected to the clinical and family support. So, for example, body temperature, changes of dietary patterns, or patterns of going to the bathroom, or sleep patterns, or loneliness, early detection of dementia, and all this aspect. But when COVID hit, one thing that caught my attention early, and this is personal to me as well, is the elderly population. I am the primary caretaker of two elderly parents who have severe preconditions, so I worried about them. As data came in, we were just watching in horror how seniors are more impacted in terms of vulnerability and possibly fatality rate and how the hospital overload creates even more problems because seniors are not only more fragile from their immune system point of view. But more of them have baseline health conditions that are now being compromised by not being able to access the healthcare system, access doctors, or going to the clinics. And suddenly we realized that what we’ve been working on in these home-based contactless sensor technology is totally relevant in this conversation. Can we accelerate this technology to bring it to the homes of seniors and the communities of seniors so that we can, on one hand, help the early detection of COVID like body temperature changes, signs of infection, as well as continue to maintain medical care and medical attention while we don’t tax the hospitals?

AZEEM AZHAR: What kind of sensors are we talking about? What data do you need for this to work?

FEI-FEI LI: So, the sensors we’ve been using and piloting, and first we are piloting hospitals, not yet homes, are two main types. One is the depth sensor. If you don’t know what that is, think about the Xbox video game where it sees humans’ movement. It uses depth information without seeing people’s faces or clothes and all that. Another kind of sensor is thermal sensor and the human behaviors we’re looking at tend to be either temperature changes or behaviors if you’re sitting on the couch for a long period of time without exercise, the frequency of food and liquid intake, sleep patterns.

AZEEM AZHAR: I’ve come across similar projects and people are often using HD cameras because they have higher resolution and you can look at things like facial ticks and twitches and you can get a closer understanding of gait. But with that high definition, of course, comes more intrusiveness in the application and its potential for misuse. What are the parameters that you put around the depth sensor?

FEI-FEI LI: Any sensor, we have to deal with the privacy issues and respect to humans. We are collaborating with ethicists and law scholars and privacy scholars, researchers on that. But having said that, the choice of depth was trying to be a little more sensitive to privacy, which means we lose data on the high fidelity and high resolution pixel coloring data. So how do we remedy that? That’s an ongoing computer vision research in our lab. We have technology that can understand the details of human postures without the RGB HD camera data.

AZEEM AZHAR: Is there always going to be a trade-off between the fidelity and the quality of data that you can get and the invasiveness potential of an application? You yourself alluded to that we’re not going to use this because we don’t want to necessarily even have access to that data. But is that a necessary trade-off? Is it a kind of fundamental axiomatic trade off in these systems?

FEI-FEI LI: That’s a great question, Azeem. I think there’s always going to be consideration. Absolutely every step of the way a technology is developed, especially human-centered technology, the human aspect, privacy, respect, dignity, it should not be an afterthought. So from that point of view, I wouldn’t even call it trade-off. It’s just part of the equation. It’s part of the consideration. It’s an opportunity for the technology to rise up so that if we cannot use certain information, it imposes us a bigger challenge and more exciting opportunity to get the technology to work without those [inaudible 00:08:06].

AZEEM AZHAR: It seems to me in this part of our conversation, we’re dancing around the institute that you have co-founded. So the institute is the Stanford Human-Centered AI Institute. And just tell me what’s different about it as an AI research institute to the group you were working with previously or other more traditional AI institutes.

FEI-FEI LI: What excites me is, from the get go, we genuinely, in our DNA have the aspiration and are working on making this a truly interdisciplinary research and education institute. The co-founder is from computer science, from philosophy, the co-founding team from economics, from law, from ethics, from medicine. I would not have dreamed 20 years ago when I began as a PhD student that this little niche field of my own intellectual curiosity will become a changing force of humanity’s future in society. And that realization personally gave me such a huge sense of responsibility and really humbling sense of responsibility that the next phase of AI cannot be void of the consideration of human condition and all aspect of our human society and individual lives.

AZEEM AZHAR: You used a particularly interesting word there that these technologies cannot be void of considerations of humanity. By implication, there has been a disconnect between what is happening in the science labs and the engineering labs, particularly in computer science and these human characterizations, and how would you articulate some of the consequences of this void?

FEI-FEI LI: And again, this is a journey of my own personal growth. I remember the very first research paper I read as a first year PhD student at Caltech was a seminal paper on face detection, and that was 2000, the year 2000. My advisor said, “Read this paper. It’s a beautiful piece of machine learning. It shows real time, using very slow CPU chips, real time face detection.” But looking back, when I was reading that paper, when I was discussing that paper in class, in labs, nobody ever, not even myself, talked about the human implication, the privacy implication. We were just excited by, “Oh wow, computer vision algorithm is capable of doing that.” So that’s a small indication of how much the field has grown because today face recognition is a big topic not only as technology, but as fairness, as privacy, as important societal topic, and it has negative consequences from racial profiling to infringing on human rights and all that. So that’s just one small example to show that the early development of these technology was void of this consideration. Was it anybody’s fault? Probably not. We never even dreamed how impactful this technology would’ve been. But today, now that we have seen these consequences, we want the future development of this technology to go much more hand in hand with these human considerations.

AZEEM AZHAR: It’s a really important observation and it’s really driven a lot of my work over the last five years, the distinction, the gap that emerges between the people working on exponential technologies and the people who work in other areas in policy, in law, in rights. There hasn’t been a common language. It’s been understood for a while. I mean, I think back to a famous lecture by C.P. Snow called The Two Cultures back from 1959, and he describes these two cultures and he describes how the average scholar who knows about Shakespeare knows nothing of the second law of thermodynamics. And the average scholar who knows about the second law of thermodynamics doesn’t know anything about Shakespeare. And he says that, “Without that joint knowledge, we can’t think with wisdom.” And this is 60 years earlier than your founding HAI in 2019, but it seems like you are constructing a bridge between those two cultures.

FEI-FEI LI: For me, it’s a double helix. That bridge has to come in so many touch points, literally like a double helix. And at Stanford we call it the next generation students should be bilingual, they should be bilingual of technology and human considerations. And in Silicon Valley, one thing that really struck me over the past few years is many young technologists telling me that they were not educated with that set of knowledge. When they now hear in the news or even look at the products their own companies are producing, and are faced with these really daunting questions of human impact, they don’t know where to even begin to think about what’s their own role, how to participate in making a difference to better the world.

AZEEM AZHAR: So, my experience in seeing less the students as researchers, but seeing them as product managers and developers and entrepreneurs is that they aren’t equipped to ask the right questions, nor are they equipped even for the sensitivity to know that that could be a question that might be asked, that might be reasonable. Now, I want to give people a flavor of HAI. You have some courses on there that really are very scientific. You’ve got knowledge graphs, theoretical neuroscience, machine learning, and causal inference. And then on the other hand, you have the politics of algorithms, ethics, public policy and technological change, digital civil society, designing AI to cultivate human wellbeing. I mean, it’s an incredible breadth of courses that are available. I am curious about what do you have to innovate in order to knit these two very disparate disciplines?

FEI-FEI LI: Most of my students are coming from a computer science background, master’s and PhD students. When they join the team that we call Partnership in AI Assistive Care, first of all, our weekly lab meeting is between five or six clinicians and the team of AI students. So, every week they’re constantly in conversation and in dialogue with the clinicians and nurses and so on. Second, every student who joins our team, the first thing we require them to do is to shadow doctors and go to the hospital and clinics. And my instruction to the students is that forget about your algorithm, forget about your equations. Just soak in the human experience, experience that human vulnerability, the human heroism, the people helping each other, people caring for each other, but also the pain, the suffering, the challenges. And I think that’s a small step towards helping students to develop that fuller body of both knowledge and experience in thinking, both in terms of technology but as human conditions. So that’s just a small example, but HAI is doing this on all fronts.

AZEEM AZHAR: It seems that there are two schools of thought. There are certain people who say that technology is ethically neutral. It is like a stone or a planet circling a star. And there are others who say that technology is in a sense, path dependent and evolves out of particular structures and contexts and biases and privileges and perspectives. And so therefore it’s never neutral. I mean, if I was to say to you, roughly speaking, which of those two views do you think holds more true where would you find yourself?

FEI-FEI LI: So, stars are not made by humans. Technologies are. And so, I do believe, and I will quote philosopher Shannon Vallor, that “There’s no independent machine values. Machine values are human values.” I think the scientific laws have their own logic, and it’s a beauty without human bias, but the innovation of technology and the application of technology is very human dependent, and we all have that ethical responsibility.

AZEEM AZHAR: So, if we think about that, it’s human dependent and humans are dependent on their context and the community around them. So when you were designing the institute, of course you are coming from the context of Stanford and you yourself are multicultural, you were born outside of the US. How do you think about the context of Stanford in of itself, nestled under the great Sequoia trees and the venture capitalists and the technology companies? Because that in itself is a particular context, perspective way of looking at the world and particular type of privilege and access.

FEI-FEI LI: Yes, and responsibility. I say that again. I think from early on, the Stanford leadership, our President, Marc Trevor Tessier-Lavigne [inaudible 00:17:47], and John H. Mundy, my co-director… we recognize this context very much. Historically, Stanford played a critical role if not a vital role in the creation of Silicon Valley. And our technologies impact the world. Our students and alumni are as we speak, some of the biggest leaders and influencers of the world. So Stanford has a historical opportunity and responsibility to think about the future of AI and what role we want to play. And this is why HAI got so much support just university-wide from hundreds of faculty and students and the leadership because we wake up and realized our role is not only innovating technology. It’s to bring that human dimension into technology and to use technology to flourish human conditions, including arts, including music, including humanities, as well as social sciences, medicine, education and so on.

AZEEM AZHAR: So I think it’s a fantastic mission. It really is. I think a lot about questions of AI bias or questions that relate to the decline of trust through computer mediated platforms. And when I was hearing what Facebook and Google were saying in 2015 and 2016, when the first image recognition systems started to go wrong, the excuses they were making were as if it was the first time they had seen these problems. But my social scientists and humanities friends would be pulling out French critical thinkers from the 1960s and saying, “Well, this person identified this as an issue.” Are there ways you’ve approached to bring that into the thinking?

FEI-FEI LI: We have so many leaders, thought leaders and scholars and teachers at Stanford making that effort. For example, Robyn Reiss’s course. Like the 300, 400 students taking that class is reading deep scholarship literature, research papers, studies, books on that topic of ethics and technology starting from 19th century and 20th century. So for example, we have this wonderful paper that just came out of Stanford with a bunch of scholars in school of education, psychology, linguistics and computer science using natural language models to understand the issue of racial gender discrimination in academia itself. We have machine learning students working with political scientists on refugee placement and how to make it more humane and effective. We have students working on poverty using machine learning, reinforcement learning, and satellite imagery. So that is happening. I think it’ll take time to see the impact of this. One thing I tell my students is be patient and be open-minded. It takes a while to acquire new language, but for those of us who are multilingual in our own life experience would know how amazing the end effect is. We get to appreciate culture, people, knowledge in a whole different way.

AZEEM AZHAR: But I’m curious about where you think the limits of these things are. One argument would be that engineers should care about these issues, but they shouldn’t set the parameters. The parameters should be set by society through some democratic process. And I suppose I’m very heartened by the progress that’s been made in the last four or five years in these areas, but I also don’t want us to fall into the trap of saying, “We’ve taught our engineers ethics and they all know a bit of [inaudible 00:21:46] and they’ve read Simone de Beauvoir and now ergo their products will be suitable.”

FEI-FEI LI: No, totally. I think that’s a fair question. First of all, I hope what we’re trying to do is trying to be humble. I think if anything we teach people or we want to teach ourselves to listen, not to think that we’ve learned something and be omnipotent and know everything. It’s to listen and to collaborate with multi-stakeholders. Our role is not to necessarily set everybody’s agenda. Our role is to help create a healthy ecosystem so that such important dialogues and brainstorming and efforts can take place. I personally hope the new generation will go out of the world as product managers and entrepreneurs and know that their team should hire an ethicist or should have ongoing conversations with vulnerable users or to work with policymakers from local to national. Instead of saying, “Because I take two courses at Stanford, I know everything.”

AZEEM AZHAR: I think it’s important that we also don’t expect everything to land on the engineer. So when we think about the healthy ecosystem that you described, what do you think we need from civil society in terms of democratic and citizen engagement and understanding of the future potentials that these technologies can create?

FEI-FEI LI: I think Azeem, it goes to the same that I think as a technologist, be humble, be open-minded, be respectful and engaged. As a technologist, I keep reminding myself how little I know about so many things and I’m not an ethics scholar. I’m not the person in the ICU bed experiencing the diseases. So I need to listen and be humble, and I do hope these kind of multi-stakeholder conversations are built upon that kind of mindset.

AZEEM AZHAR: You’ve quoted Shannon Vallor said, “There are no independent machine values, but machine values are human values.” Can we look at one particular question? So for example, we’ve seen that many systems built on algorithms have acted not through design to diminish trust in society. What would you say we would need to do in order to construct systems that increased rather than diminish trust?

FEI-FEI LI: Our algorithms are by and large right now, black boxes. They’re hard to be interpreted. They’re not explainable. The robustness and the safety constraints are not well parameterized and understood, and that erodes trust and is a shaky foundation that we need to solidify. So a lot of theorists and theoretical computer scientists, statisticians and machine learning researchers are now working on that very problem. And this includes bias and all that. The whole algorithm design and human interface is another huge issue. This is why I keep saying all these human issues cannot be afterthoughts. They have to be baked into the design of a technical system. And that starts from where you get data. How do you annotate data? How do you use the data? How do you interpret the results? How do you build in human factors? How do you enable humans to interact with the machine learning or AI system when it has conflict or needs to collaborate with each other? And the third is everything I said should be embedded in ethical frameworks that technologists alone cannot come up with. We need the scholars like you trained in social science and ethics and philosophy to work with us.

AZEEM AZHAR:

Do you think that a lot of those considerations need to then also be reflected in law so that the guardrails are firmly in place for monitoring and in compliance and enforcement of these particular rules that you can identify?

FEI-FEI LI: One of the most exciting things I’m learning as just my knowledge is so minimal in this journey of HAI is the law aspect, the policy and regulatory law. There is tremendous movement and the effort and excitement in the legal world about AI and machine learning. You’re totally on the spot. Law and regulatory policies have to participate in this conversation.

AZEEM AZHAR: If I may, I’d like to turn to something else, which was your project ImageNet, which I think was so critical in creating this boom that we’re seeing today in an investment and application of artificial intelligence. It reminded people of the importance of data to these systems. Did you imagine that this might be the impact of the project when you got started in 2006?

FEI-FEI LI: What excited me and most of scientists is the intellectual pursuit and the curiosity and the hypotheses we formed. Less about how big the impact it would be from a societal angle.

AZEEM AZHAR: Right. It is remarkable. Essentially you did a lot of heavy lifting to classify images, which created a clean data set that people could then throw their algorithms on. And in the years running up to 2012, it was about $300 million a year going into AI startups. And then after 2012, and for the next 7 years, it was a 60% annualized growth in the amount of money that went in just from venture capitalists, not to mention the tens of billions that went in from companies. And the thing that’s remarkable is you can literally see the overlay, the graph of ImageNet competition performance with the rate with which funding increased. So it had this huge, huge impact. But it feels to me that, and forgive me if I’m being too direct here, that your part in that story is not as perhaps well understood as maybe I feel it is.

FEI-FEI LI: Well, first of all, I’m humbled you say that, and I appreciate you giving credit to [inaudible 00:28:26] that. I think I’ll let history and time tell and judge in terms of our contribution. But we’re definitely very proud of ImageNet.

AZEEM AZHAR: So, if I think back to where image recognition was in 2011, 2012, it was much better than it was in 2009, but it was nowhere near as impressive as we see today by the combination of better data and optimizations in the algorithms and more processing. Sometimes then people looking outside who are non-experts looking at this, think that this is a really radical breakthrough, it is a represent of a paradigm shift. Now, as a scientist in the field, how do you interpret what’s gone on over the last six years?

FEI-FEI LI: ImageNet was born out of this desire, we need a radical shift. Our hypothesis at that time is not too different from a lot of scientific discovery, is that we need to establish a North Star that can truly drive the research of visual intelligence. And that North Star is a combination of defining the right problem, which ImageNet is about object categorization at large scale, and creating that path to achieve that problem, which was through big data. And what I think we succeeded in doing is creating that North Star. Again, we were stepping on the shoulders of giants. We didn’t pull out that North Star out of thin air. There was 30 years of cognitive neuroscience research and computer vision research that was fueling that thinking. But we did in a way definitively define that North Star and established a critical path, which is that in order to reach that North Star machine learning at that time, need to go through supervised learning with big data. And with the Morse law carrying the chip advancement, which was a parallel development and internet creating data at a scale, humanity had never seen, these three forces converged, and this is what brought that paradigm shift.

AZEEM AZHAR: And we can now recognize a cat from almost any angle, if you like.

FEI-FEI LI: Exactly.

AZEEM AZHAR: Again, for people who are outside of the industry, the question that has been raised more and more in the popular media has been this idea of artificial general intelligence or AGI. What does AGI mean to you?

FEI-FEI LI: When I read Turing’s original Dreams for Thinking Machines and the founding fathers, they’re not that different from what AGI in the popular media stands for, which is that nuanced, complex, contextualized capability of perceiving, cognizing, and reasoning and doing by artificial systems, whether it is playing chess, or go, or cooking omelet or helping elderly to have a conversation. All these are part of the technical aspirations of AI since the dawn of this field. So from that point of view, I think AGI was created to emphasize.

AZEEM AZHAR: I’m curious about how when we think about AI and AGI, whether we don’t fall into a couple of anthropocentric traps. The first being that we can simply engineer our way to this. The second is the idea that we even could understand the potential of what a machine intelligence could be from our own frame of the limits of human intelligence.

FEI-FEI LI: So, I think scientists should be allowed to dream big. When Newton was looking at the stars, humanity didn’t even have electricity yet, but the path to that and the consequence, I do believe we need humility. I think it’s really exciting from a intellectual point of view is the effort of many people is the cross-pollination of AI and brain sciences and cognitive sciences. So at Stanford, in fact, one of the three intellectual principles of HAI is that human inspired intelligence and the arrow goes both ways to neuroscience and come back to AI, is how we can also continue to work to advance our understanding of human cognition and human brain and in turn, learn more from it to improve AI.

AZEEM AZHAR: The ability of some of the vision algorithms that you have helped pioneer and develop to resolve bits of the real world now exceeds much of human vision because we can feed in multi-spectral data from frequencies that I can’t see at a much higher resolution than my eye can pick up. And so you can with an algorithm, do much more than a human ever could. That’s just in a narrow application. When you look out 15, 20, 30 years, do you think our AI systems, the intelligence, their ability to make decisions given their environment will look similar, feel similar to a humans’ map and model of intelligence?

FEI-FEI LI: I think there is definitely a scientific curiosity collectively that how do we continue to push for creating innovating machines that can go towards that kind of mental image of human intelligence? For one thing, if we could do that, we can help to interface with humans better and help humans better, in many ways, if our machines understands humans and can think like humans. But again, this development cannot be void of all the guardrails that you have been talking about. Humanity has always been innovating, out-innovating ourselves. We no longer run fast compared to our cars and even horse carriages. If you look at the tools we created, it’s all about extending and augmenting ourselves and sometimes going directly way beyond what we can do and sometimes just replicating so that we have more hands to help. But no matter which side of that is, the guard rails has to be there while we advance this technology.

AZEEM AZHAR: Fei-Fei Li, those are very, very wise words. Thank you so much for taking the time to speak with me today. I appreciate it.

FEI-FEI LI: Thank you, Azeem. That was really enjoyable. Thank you.

AZEEM AZHAR: Well, thank you for listening to this podcast. I really hope you enjoyed it. There are a ton of others that you can listen to with other AI experts. Just look through the archive. I’m sure you’ll find them and love them. This podcast was produced by Marija Gavrilov and Fred Casella. The researcher was Ilan Goodman. Bojan Sabioncello is a sound editor. My name is Azeem Azhar and this is a production of E to the Pi i Plus 1 Limited. Thank you very much.