Yann LeCun is a recipient of the Turing Award and a leading AI researcher, yet he has consistently taken a contrarian stance in the technology sector. He argues that the current fixation on large language models within the industry is misguided and will ultimately fail to address numerous urgent issues.
Rather, he believes we should invest in world models—a different kind of AI that accurately mirrors the complexities of the actual world. He is also a fervent proponent of open-source AI and critiques the secretive methodologies of pioneering labs such as OpenAI and Anthropic.
Perhaps it is not surprising that he recently departed Meta, where he served as the chief scientist for FAIR (Fundamental AI Research), the influential research laboratory he established. Meta has faced difficulties in gaining traction with its open-source AI model Llama and has experienced internal upheavals, including the contentious acquisition of ScaleAI.
LeCun spoke with MIT Technology Review in an exclusive online conversation from his Paris apartment to elaborate on his new endeavor, life post-Meta, the future of artificial intelligence, and his views on why he believes the industry is pursuing flawed concepts.
Both the inquiries and responses below have been modified for clarity and conciseness.
You’ve just unveiled a new enterprise, Advanced Machine Intelligence (AMI). What are the key concepts behind it?
It will be an international company, but based in Paris. You pronounce it “ami”—which translates to “friend” in French. I’m thrilled. Europe boasts a high concentration of talent, yet it often lacks the conducive environment for growth. Moreover, there is indeed significant demand from industries and governments for a reputable frontier AI firm that is neither from China nor the US. I believe this will work in our favor.
So, it’s a bold alternative to the current US-China dichotomy. What motivated you to pursue this third avenue?
Many countries face sovereignty concerns and desire some level of control over AI. My position is that AI will evolve into a platform, and most platforms tend to become open-source. Unfortunately, that’s not really the path being taken by the American industry. Right? As competition intensifies, they feel compelled to be secretive. I believe this is a strategic blunder.
This is certainly true for OpenAI, which transitioned from being very open to extremely closed, while Anthropic has always maintained a closed stance. Google was somewhat open at one time. And as for Meta, we’ll have to see. My impression is that it’s not advancing in a positive way right now.
In contrast, China has fully embraced this open strategy. Consequently, all major open-source AI platforms are Chinese, leading to a scenario where academia and startups outside the US primarily adopt Chinese models. There’s nothing inherently wrong with that—after all, Chinese models are effective, and their engineers and scientists are exceptional. However, if our future is dictated by AI tools that filter our information, and the choices are between English-centric models from proprietary firms closely tied to the US or Chinese models that may be open-source but require nudging to address topics like Tiananmen Square in 1989, then it’s not exactly a desirable future.
Those future models should have the ability to be fine-tuned by anyone and yield a broad diversity of AI support, incorporating various linguistic capabilities, value systems, political biases, and interests. A high diversity of assistance is essential for the same reason a diverse press is necessary.
This is certainly an intriguing argument. How are investors responding to this vision so far?
They are very enthusiastic. Many venture capitalists strongly support this notion of open-source, as they acknowledge that many smaller startups heavily depend on open-source models. They lack the resources to develop their own models, making it strategically risky for them to rely on proprietary models.
You recently exited Meta. What’s your perspective on the company and Mark Zuckerberg’s leadership? There’s a view that Meta has squandered its AI edge.
I believe FAIR [LeCun’s lab at Meta] was remarkably successful in research endeavors. However, where Meta fell short was in translating that research into practical technologies and products. Mark made decisions he believed were in the company’s best interest, though I may not have concurred with all of them. For instance, the robotics group at FAIR was dismantled, which I consider a strategic misstep. But I’m not the head of FAIR. Decisions are made logically, and there’s no point in harboring resentment.
So, no ill feelings? Could Meta be a potential future client for AMI?
Meta could indeed be our first client! We’ll see. Our work is not in direct competition. Our emphasis on world models for the physical realm differs significantly from their focus on generative AI and LLMs.
You were engaged with AI long before LLMs became a mainstream approach. Yet, since the rise of ChatGPT, LLMs have become almost synonymous with AI.
Indeed, and we intend to change that. The public perception of AI primarily revolves around LLMs and various types of chatbots. However, the latest iterations are not purely LLMs; they integrate LLMs with other elements such as perception systems and specialized problem-solving code. Therefore, we will view LLMs as somewhat of an orchestrator within broader systems.
Beyond LLMs, a significant amount of AI operates behind the scenes, impacting vast segments of our society. Examples include assisting with driving a vehicle, generating quick MRI images, and algorithms governing social media—all of this constitutes AI.
You’ve been vocal in claiming that LLMs have their limitations. Do you think they are currently being overstated? Can you clarify for our readers why you think LLMs fall short?
In some ways, they haven’t been overstated, particularly as they are incredibly beneficial for many people, especially those involved in writing, research, or coding. LLMs excel at manipulating language. However, there’s a prevailing illusion, or perhaps a delusion, that it’s merely a matter of time before we can enhance them to achieve human-level intelligence, which is simply untrue.
The genuinely challenging aspect lies in grasping the real world. This phenomenon, known as the Moravec Paradox (observed by computer scientist Hans Moravec in 1988), illustrates that what we find easy—like perception and navigation—poses significant challenges for computers, and vice versa. LLMs operate within the discrete realm of text; they cannot genuinely reason or plan due to a lack of a real-world model. They fail to forecast the ramifications of their actions. This is precisely why we lack domestic robots that are as nimble as house cats or truly autonomous vehicles.
We will eventually develop AI systems with human-like and human-level intelligence, but they won’t be founded on LLMs, and this won’t happen next year or in the subsequent two years. It will require time. Major conceptual breakthroughs are essential before we realize AI systems with human-level intelligence. And that’s what I am focusing on. This new venture, AMI Labs, aims for the next iteration.
Your solution consists of world models and JEPA architecture (JEPA, or “joint embedding predictive architecture,” is a learning framework that trains AI models to comprehend the world, created by LeCun during his tenure at Meta). What’s the selling point?
The future is inherently unpredictable. Attempting to create a generative model that can forecast every nuance of what lies ahead will ultimately lead to failure. JEPA isn’t generative AI; it’s a system designed to learn to represent videos exceptionally well. The crucial factor is understanding an abstract representation of the world and making predictions within that abstract framework, disregarding the unpredictable specifics. That’s the essence of JEPA—it learns the fundamental rules of the world through observation, akin to a child learning about gravity. This foundation is key to common sense and crucial for constructing genuinely intelligent systems capable of reasoning and planning in the real world. The most exhilarating advancements in this domain are currently emerging from academia, rather than the large industrial labs bogged down in the LLM arena.
The absence of non-text data has impeded the advancement of AI systems in understanding the physical realm. JEPA is trained on videos. What other forms of data will you utilize?
Our systems will be developed using video, audio, and various types of sensor data—not just text. We are experimenting with different modalities, from robot arm positioning to lidar data and audio. I am also involved in a project utilizing JEPA to model intricate physical and clinical phenomena.
What tangible, real-world applications do you foresee for world models?
The possibilities are extensive. Consider intricate industrial processes with thousands of sensors, such as in a jet engine, a steel production facility, or a chemical plant. Currently, there is no method to construct a comprehensive, holistic model of these systems. A world model could learn from sensor data and forecast the system’s behavior. Alternatively, envision smart glasses that observe your actions, recognize what you’re doing, and then predict your subsequent actions to assist you. This is what will ultimately render agentic systems dependable. An agentic system intended to take actions in the world cannot function reliably without a world model to foresee the implications of its actions. Without it, the system is bound to make errors. This is pivotal for unlocking everything from genuinely functional domestic robots to Level 5 autonomous driving.
Humanoid robots are recently gaining much attention, especially those created by companies from China. What are your thoughts on this?
Many approaches are being employed to sidestep the constraints of learning systems, which necessitate massive datasets for effective execution. Thus, the success of companies developing robots to perform kung fu or dance lies in extensive pre-planning. Honestly, no one—absolutely no one—has figured out how to imbue these robots with sufficient intelligence to be genuinely useful. Trust me on this.
Each task requires a vast quantity of teleoperation training data, and when conditions shift even slightly, the robots struggle to generalize. This hints at our profound shortcoming. The fact that a 17-year-old can master driving in 20 hours is due to their pre-existing knowledge of how the world functions. For a truly versatile domestic robot, we need systems that understand the physical world well. That will remain unattainable without solid world models and planning.
There’s an increasing belief that foundational AI research is becoming more challenging in academia due to the extensive computing power needed. Do you think that crucial innovations will now stem from industry?
No. LLMs are now a matter of technological advancement, not research. While it’s accurate that academics face significant hurdles in making meaningful contributions due to the hefty demands for computation, data accessibility, and engineering support, it has become a product—not a topic worth academic interest. It’s akin to speech recognition in the early 2010s—it was a resolved issue, with progress resting in the hands of the industry.
Academia should focus on long-term goals that transcend the capabilities of existing systems. That’s why I advise people in universities: Avoid LLMs. It’s futile. Competing with industry efforts is unrealistic. Pursue alternative avenues. Innovate new methodologies. The breakthroughs won’t originate from merely scaling up LLMs. The most exciting advancements in world models are surfacing from academia, not the major industrial labs. The concept of applying attention circuits in neural networks originated at the University of Montreal, which sparked the entire revolution. As established companies tighten their closures, the rate of breakthroughs will decelerate. Academia requires greater access to computational resources, but their emphasis should be on the next major breakthrough, not on fine-tuning the last one.
You hold multiple roles: professor, researcher, educator, public thinker … Now, you’ve taken on a new one. How will that shape your future?
I will serve as the executive chairman of the company, while Alex LeBrun [a former colleague from Meta AI] takes the role of CEO. It’s going to be LeCun and LeBrun—it sounds pleasant when pronounced in the French style.
I plan to retain my position at NYU. I teach a single class annually, mentor PhD students and postdocs, so I will remain anchored in New York. However, I frequently travel to Paris for my lab work.
Does this imply that you won’t be very hands-on?
Well, there are two interpretations of being hands-on. One involves daily management, while the other entails immersing oneself in research efforts, right?
I can handle management, but it’s not what I relish. My true mission is advancing science and technology as far as possible, inspiring others to engage in meaningful endeavors, and contributing to those pursuits. That has characterized my role at Meta for the past seven years. I founded FAIR and directed it for about four to five years. I found being a director unappealing; I’m not suited to this career management aspect. I’m far more of a visionary and a scientist.
What qualifies Alex LeBrun as the ideal choice?
Alex is a serial entrepreneur; he has built three successful AI startups. The first was sold to Microsoft, the second to Facebook, where he led the engineering division of FAIR in Paris. He then left to establish Nabla, a highly successful enterprise in healthcare. When I approached him about joining me on this venture, he accepted almost at once. He possesses the expertise required to build the company, allowing me to concentrate on scientific and technological advancement.
Your headquarters is in Paris. Are you planning to establish offices elsewhere?
We are an international company. An office will be set up in North America.
New York, perhaps?
New York is wonderful. That’s where I am, isn’t it? Plus, it’s not Silicon Valley, which feels a bit like a monoculture.
What about Asia? I assume Singapore is on the list?
Most likely, yes. I’ll let you deduce.
And how are you attracting talent?
Recruitment is not a challenge for us. Many individuals in the AI research community believe the future of AI is rooted in world models. Those people, regardless of salary, will be eager to join us because they are passionate about the technological future we are striving to build. We’ve already brought on board talent from organizations such as OpenAI, Google DeepMind, and xAI.
I’ve heard that Saining Xie, a distinguished researcher from NYU and Google DeepMind, may be joining you as chief scientist. Any insights?
Saining is an exceptional researcher. I hold him in high regard. I have already hired him twice—once at FAIR and also convinced my colleagues at NYU to recruit him. Let’s just say my respect for him is substantial.
When will you be ready to disclose more details about AMI Labs, such as financial backing or other key members?
Soon—in February, perhaps. I’ll keep you posted.
