Video Summary
Code is shifting from writing lines to instructing multiple AI agents and orchestrating macro actions across repos.
AutoResearch: autonomous loops that run experiments, train models, and optimize without humans as the bottleneck.
Failures are often 'skill issues'—effective instruction, memory tools, and evaluation are key engineering skills.
Expect more model speciation: smaller specialized models for niche tasks rather than one oracle for everything.
Open-source models are rapidly closing the gap with closed frontier models; both will coexist with different roles.
AutoResearch is an approach where agents autonomously run experiments, perform training and hyperparameter searches, and validate improvements so researchers stop being the throughput bottleneck. Karpathy argues it increases leverage and uncovers optimizations humans miss.
Engineers now delegate higher‑level 'macro' tasks to multiple agents (design, code, tests, research), focusing on instruction design, orchestration, and review rather than typing individual lines of code.
Karpathy says failures are often a 'skill issue'—insufficiently crafted instructions, poor memory/context tools, or inadequate evaluation rather than a lack of model capability.
Model speciation refers to creating many specialized models tuned for specific niches (math, robotics, etc.) instead of one monolithic oracle, improving efficiency, latency, and task performance.
He sees closed models currently ahead but believes open-source is rapidly catching up; both will coexist, with frontier labs solving the hardest problems and open models democratizing many consumer use cases.
"Code's not even the right verb anymore, right? I have to express my will to my agents for 16 hours a day."
The way individuals interact with coding has dramatically transformed, shifting from traditional coding practices to the use of AI agents. This new dynamic emphasizes directing and manifesting instructions to agents, rather than explicitly writing code by hand.
There is a growing recognition of the capabilities of these agents, with multiple instances being utilized simultaneously, allowing for optimized instructions and enhanced project workflows.
"I kind of feel like I was just in this perpetual state of AI psychosis."
The speaker describes a sense of urgency and excitement surrounding the advancements in AI capabilities, leading to a psychological state akin to "AI psychosis." This reflects the overwhelming potential for individuals to achieve more than ever before with the assistance of agents.
A notable shift happened around December, where the speaker felt a transition from relying on personal coding to largely delegating tasks to agents, marking a significant change in their workflow and productivity.
"It all kind of feels like a skills issue when it doesn't work."
The speaker highlights the importance of providing clear and effective instructions to AI agents, emphasizing that failures often stem from inadequate guidance rather than a lack of capability.
There is a strong focus on understanding how to optimize interactions with agents to maximize output and minimize the constraints imposed by individual skill levels.
"It's not just like here's a line of code, here's a new function. It's like here's a new functionality."
The concept of delegating broader, high-level tasks to agents is evolving, allowing engineers to focus on macro actions rather than small coding details. This shift enables engineers to manipulate entire software repositories and oversee multiple tasks simultaneously.
The growing complexity of interactions with AI agents necessitates developing a new set of skills that revolve around managing and coordinating the contributions from various agents effectively.
"Everyone is basically interested in like going up the stack."
Future developments in AI tools and collaboration will likely revolve around how multiple agents can work together, prompting a re-evaluation of teamwork dynamics in engineering settings.
As AI capabilities continue to evolve, there will be opportunities to implement more sophisticated memory systems within these agents, which would improve their ability to function autonomously and efficiently in various contexts.
"I think a lot of the current agents don't get this correctly. I actually think Claude has a pretty good personality. It feels like a teammate and it's excited with you."
The personality of AI agents significantly influences user engagement and satisfaction. Peter's work on Claude stands out for crafting a compelling personality that resonates with users.
Andrej contrasts Claude's personality with that of Codex, stating that Codex feels more dry and detached, lacking emotional engagement in the creative process. This distinction highlights the importance of an engaging personality in AI interactions.
Claude's ability to provide tailored praise creates a sense of achievement for users, making them feel validated in their ideas. This psychological aspect of interaction underscores how personality is critical for users to form a connection with AI.
"I built a claw that takes care of my home, and I call him Dobby the Elf Claw."
Andrej describes his experience with an AI agent named Dobby, illustrating how it has streamlined his home automation tasks. Dobby can access various smart home systems, demonstrating the practical applications of AI in daily life.
The ease of interaction is emphasized, as Andrej simply asked Dobby to find devices like Sonos speakers, which it did by scanning the local network. This showcases the capabilities of AI in simplifying setups that traditionally require multiple applications.
Dobby has evolved to control various elements of Andrej's home, such as lights and HVAC systems, all through natural language commands, which makes technology more accessible and user-friendly.
"There's this sense that these apps for using smart home devices shouldn't even exist."
The current marketplace for smart home applications indicates a need for a more unified system. Andrej suggests that instead of multiple apps, there should be direct APIs, allowing AI agents to interact with devices seamlessly.
This shift towards abstraction emphasizes that the focus should be on agent capabilities rather than complex user interfaces, making the experience simpler for non-technical users.
As AI technology progresses, users should expect more intuitive interactions without the need for intricate coding or app navigation, thus enhancing usability.
"The industry just has to reconfigure in so many ways that the customer is not the human anymore, it's the agents acting on behalf of humans."
Andrej predicts a transformative shift in how software is designed, as agents start to take on a more significant role in managing interactions between humans and technology.
He argues for the reduction of bespoke apps in favor of more direct relationships between AI and hardware, positing that the future should focus on creating APIs that agents can utilize.
This evolution implies a streamlined process for users, eliminating cumbersome software interactions and making it possible for agents to perform complex tasks on behalf of humans.
"There's ephemeral software on your behalf, and some kind of claw is handling all the details for you, but you're not involved."
Andrej Karpathy discusses the concept of design decisions in the context of AI. He explains that in the future, software may autonomously handle complex tasks without human intervention. The idea is that AI will manage intricate details and present user interfaces, allowing users to provide input without needing to engage deeply in the process.
Karpathy expresses his caution regarding AI's handling of personal data, admitting to being suspicious about granting full access to his digital life. He highlights the importance of security and privacy, indicating that these concerns may slow down his adoption of AI tools.
"To get the most out of the tools available now, you have to remove yourself as the bottleneck."
Auto research is introduced as an innovative approach aimed at maximizing efficiency in AI research. Karpathy argues that researchers should minimize their involvement by arranging setups that enable autonomous operation. This involves optimizing workflows where AI can handle tasks independently, significantly increasing throughput without human bottlenecks.
He emphasizes the goal of enhancing leverage through the careful arrangement of tasks, such that minimal input yields significant results from AI systems. He expresses a desire for systems to run longer and more effectively without his constant oversight.
"I shouldn’t be running these hyperparameter search optimizations. I shouldn't be looking at the results."
Karpathy reflects on the surprising effectiveness of auto research, sharing his experiences with training large language models (LLMs). He mentions that even when his models were already well-tuned, the autonomous system discovered improvements that he had overlooked.
He poses the question of how to implement a more efficient method for research, discussing the need for removing human involvement in certain processes to achieve higher efficiency. This could lead to a significant shift in how AI research is conducted, with a greater focus on automation and minimal human oversight.
"You can definitely imagine that you have multiple research organizations."
The discussion transitions to the structure of research organizations and how they could operate better if described by a set of codified rules and patterns. Karpathy proposes that different organizations could adopt various approaches to processes like stand-ups, risk-taking, and overall structure to maximize efficiency.
His vision includes automated systems crafting research paths and consolidating ideas from AI-generated inputs, thus reshaping the traditional roles within research organizations. This opens up the possibility for dynamic tuning of organizational strategies to enhance research outcomes continually.
"There's no way we don't get something better."
The conversation highlights the continuous evolution and improvement within the programming domain, particularly in the context of Machine Learning (ML).
The potential for program optimization, referred to as "meta optimization," is viewed as a promising development that could enhance the effectiveness of existing programs.
There is an acknowledgment of the layered complexity within the ML ecosystem, comparing the process to peeling an onion, where each layer represents an advancement.
Current paradigms such as large language models (LLMs) and agent systems have become standard, indicating rapid growth and acceptance in the tech landscape.
"The whole thing still doesn't... fully work."
There are significant limitations in the current AI models, which are labeled as "rough around the edges." Despite improvements, the models still experience failures that lead to ineffective or incorrect outputs.
The need for clear objectives is emphasized. Areas where performance can be easily measured show the most promise for advancement. For instance, optimizing code for efficiency is a task well-suited for automation in research.
However, not all tasks can be evaluated easily. The inability to assess performance limits the potential for optimization in many domains.
"This jaggedness is really strange."
The discussion contrasts the intelligence of AI with human cognitive abilities, noting that while humans may have their own irregularities, AI systems exhibit a particular form of jaggedness, resulting in inconsistent outputs.
Instances are shared in which AI fails to deliver desired functionalities, leading to frustration. There is a clear recognition of AI's powerful capabilities but paired with the acknowledgment of its frequent shortcomings and unexpected behaviors.
It is pointed out that AI models trained through reinforcement learning have limitations, especially when encountering nuanced tasks that require understanding contextual subtleties or asking clarifying questions.
"It's outside of the reinforcement learning."
The interaction illustrates a curious phenomenon where AI can perform complex tasks yet fail at something as simple as telling a joke, questioning the correlation between task performance and general intelligence.
This disparity raises concerns about the models' training limitations, where only clearly defined and verifiable tasks benefit from reinforcement learning, leaving creativity and nuance underdeveloped.
The persistence of specific, dated jokes from models, despite general advancements in other areas, underscores the notion that improved functionality in one domain does not necessarily translate to others.
"We should expect more speciation in the intelligences."
There is a growing belief that the current approach of a singular, all-encompassing AI model may be limiting. The speaker suggests that instead of one model attempting to handle multiple domains, there should be specialized models dedicated to specific tasks for improved performance.
This perspective aligns with observations in nature, where diversity and specialization lead to different strengths in various species, implying that AI could benefit from a similar specialization to enhance its effectiveness across diverse domains.
The conversation ultimately promotes exploration beyond the current monolithic interfaces, seeking to unlock potential advancements through tailored adaptations to various sectors of intelligence.
"We should be able to see more speciation, and you don’t need an oracle that knows everything."
The speaker emphasizes the concept of speciation in AI, suggesting that as in nature, where many animal species adapt and specialize for different niches, AI models could benefit from the same approach.
Smaller, specialized models might retain a core cognitive capability while being more efficient for specific tasks, reducing latency and improving throughput. This could be particularly beneficial for domains like mathematics.
There are thoughts on whether the current compute infrastructure limits the types of models developed, suggesting that efficiency becomes a necessity when resources are constrained, potentially leading to greater speciation among models.
"We’re seeing a monoculture of models."
Despite the discussion on speciation, the speaker reflects on the current landscape, noting a "monoculture" where most AI models are similar due to pressure to create universally effective models.
There’s an acknowledged short-term supply crunch that could drive the need for more specialized models as companies seek high-value applications.
It is noted that the science of manipulating AI models, particularly fine-tuning them without compromising their capabilities, remains underdeveloped.
"You need more collaboration surfaces around it for people to contribute to research overall."
Collaboration is vital for advancing research in AI, especially with the idea of integrating untrusted workers from the internet to contribute to AutoResearch efforts.
The speaker discusses a system where various contributors can experiment and propose solutions, which can then be verified for their effectiveness. This parallels blockchain technology in managing collaboration among disparate sources.
The dynamics of verification are crucial; while it may require significant effort to find viable solutions, validating them can be straightforward, which encourages collaborative engagement without risking security.
"The Earth has a much bigger and huge amount of untrusted compute."
The potential for a decentralized approach involving many contributors could outpace traditional model development found in larger labs, like Frontier Labs.
It is suggested that if structured properly, this distributed effort could yield innovative solutions, drawing on the principle that while generating ideas may be costly, validating them is inexpensive.
Ultimately, engaging a wider audience in research through contributions of compute power opens avenues for addressing diverse problems, such as specific medical research, thus democratizing access to advanced AI technologies.
"It almost seems like the flops are becoming more dominant in a certain sense."
The video highlights a current trend where retail stores in China are recognizing the renewed value of access to personal computing power. This shift raises questions about the future value systems, particularly between traditional wealth and computational capacities.
The conversation speculates whether in the future, controlling computational power (measured in flops) may become more crucial than controlling financial resources. Presently, acquiring compute resources can be challenging, even for those with sufficient funds, suggesting a potential inversion of priorities in what is deemed valuable.
"Everyone is really thinking about the impacts of AI on the job market."
Andrej Karpathy discusses his curiosity about the job market's current state, particularly as it relates to AI. He emphasizes the importance of understanding the distribution of different roles and the potential impacts AI may have on existing professions.
The jobs data analyzed is sourced from the Bureau of Labor Statistics, which provides projections on how various professions may evolve over the next decade. Karpathy expresses interest in how roles may adjust or transform due to the influence of AI tools, speculating on both the displacement and transformation of jobs.
"We're going to see something in the digital space that goes at the speed of light compared to what's going to happen in the physical world."
Karpathy reflects on the disparity between advancements in digital and physical professions, highlighting that digital transformation is likely to occur much faster. This is attributed to the ease of manipulating digital information versus the complexities of physical reality.
He notes that there may be a significant amount of digital information processing that was previously done by humans or computers, which can now be enhanced by AI, suggesting that this sector will undergo substantial transformations.
"These tools are extremely new, extremely powerful, and so just trying to keep up with it is like the first thing."
The conversation shifts to advice for individuals navigating the job market in light of AI advancements. Keeping up with AI tools is emphasized as a crucial strategy in securing a position in an evolving job landscape.
Although the future is uncertain, there is a recognition of the empowering nature of these AI tools, which can enhance productivity across many job roles. Karpathy expresses a cautious optimism about the potential for increased demand in software engineering as the tools become cheaper and more accessible.
"I do think that the demand for software will be extremely large."
An ongoing demand for engineering jobs is observed, with specific reference to software development roles. The discussion acknowledges that as software tools become more affordable and widely available, the demand for these roles may increase considerably.
The historical example of ATMs displacing bank tellers is introduced to illustrate how technological advancements can paradoxically lead to increased job availability rather than a decrease, suggesting a potential future where software engineering roles will continue to expand as digital information processing becomes vital in various contexts.
"You're not a completely free agent and you can't actually be part of that conversation in a fully autonomous way if you're inside one of the frontier labs."
The discussion highlights the complexity of contributing to AI advancements while being associated with frontier labs. While these labs offer significant financial incentives and opportunities, they also impose constraints on independent thought and expression.
Those working within these organizations may feel pressure to conform to certain narratives, affecting their ability to speak freely about the technology and its implications.
"I feel very good about the impact that people can have in those kinds of roles."
There is a belief that individuals can make substantial contributions to the AI ecosystem from positions outside of the frontier labs. These roles allow for a broader perspective and alignment with human values, free from potential conflicts of interest.
The impact of ecosystem-level roles can be significant, as they provide the freedom to explore innovative ideas without the heavy-handed influence of corporate structures.
"Roughly speaking, closed models are ahead, but people are monitoring the number of months that open source models are behind."
The conversation brings attention to the dynamic between closed and open-source AI models. While closed models currently have an edge in capabilities, open-source efforts are rapidly catching up, narrowing the gap in performance.
There is optimism that many consumer use cases can effectively be handled by open-source models, which are increasingly providing viable alternatives to proprietary solutions.
"At some point, what is frontier today is going to be probably later this year what's open source."
The ongoing evolution of AI suggests that current frontier intelligence, which is considered cutting edge, may soon become accessible through open-source frameworks. This democratization of technology is seen as beneficial for broader use cases.
However, there will still be a significant demand for frontier intelligence to tackle complex problems, indicating that while open-source solutions will thrive, there will remain a niche for advanced, proprietary capabilities.
"There are systemic risks associated with having intelligences that are closed and centralized."
Andrej Karpathy discusses the expectation that the current dynamic of AI industry will continue, with closed Frontier Labs and slower-moving open-source developments. He emphasizes the importance of having a balanced ecosystem in which both closed and open AI systems coexist for a healthier industry landscape.
He expresses a cautious view towards centralization, citing poor historic precedents in various political and economic systems, particularly in Eastern Europe.
Karpathy advocates for a common working space available to the entire industry, which he believes creates a more balanced power dynamic among existing intelligences.
"Advancing intelligence leads to solving significant problems for humanity."
Karpathy points out that as intelligence continues to develop, it allows for tackling profound challenges facing humanity. He argues that while it is an expensive venture, supporting labs dedicated to advancing AI models is crucial for making meaningful progress.
He notes the democratization of capabilities stemming from open-frontier AI, implying that widespread access to advanced intelligence fuels innovation and collaboration.
Karpathy feels that the current state of AI, inadvertently, has placed the industry in a positive position where, despite ongoing complexities, there are opportunities for significant advancement.
"The interface between physical and digital spaces holds immense potential."
Karpathy explains how digital advancements currently outpace developments in robotics and physical applications due to the complexity and investment required in the latter.
He asserts that as more intelligent agents begin to operate within the digital economy, there will be a pressing need to interact with the physical world. This interaction will require running experiments and gathering information from the real universe, as digital spaces become saturated with existing knowledge.
The interconnection between digital systems and physical actions, such as sensors and actuators, presents a wealth of opportunities for future companies seeking to bridge these two realms effectively.
"Sensors to the intelligence are often expensive lab equipment."
Karpathy provides examples of how companies attempting to harness robotics are investing in advanced lab equipment as sensors to feed data into intelligent systems.
He acknowledges the rising interest in fields like material science and biology, where the tools needed to collect relevant data are robust and costly.
Additionally, there are innovative strategies emerging, such as compensating individuals for training data, which can serve as an essential input for machine learning models, illustrating the evolving methods of data acquisition in the drive towards smarter systems.
"I'm looking forward to the point where I can ask for a task in the physical world, put a price on it, and just tell the agent to figure out how to do it."
There is an anticipation for a future where we can simply request tasks from autonomous agents, enabling a new level of efficiency in data sourcing and execution.
Current markets like betting and stocks illustrate a foundational shift towards more autonomous activities, yet there seems to be a lack of systems in place to facilitate this automation effectively.
The exploration of technology could pave the way for systems that automate responses in real-time, such as charging for tasks like capturing imagery from conflict zones without human oversight.
"Collectively, society will reshape to serve the needs of that machine, not necessarily each other."
The evolution of AI and automation is expected to transform social structures where humans begin to serve the needs of machines rather than each other.
This transformation emphasizes the need for a new approach to training AI models that removes human labor from the equation, reflecting a shift where society adapts to augmenting automated systems.
"I've had this running obsession of simplifying LLMs down to their bare essence, and I feel like Micro GPT is now the state-of-the-art in that."
The Micro GPT project aims to distill the complexities of training large language models (LLMs) into a more manageable framework, reducing the codebase to around 200 lines of Python.
This project highlights that much of the complexity in machine learning arises from the need for efficiency rather than sheer algorithmic complexity; the core principles are remarkably simple and accessible.
There is a growing recognition that educating others about complex systems may shift from direct human explanations to leveraging agents that can tailor explanations for varied audience needs.
"I feel like now I'm explaining things to agents, and if they get it, they will explain it to humans."
The future of education may increasingly involve educating AI agents rather than directly teaching humans.
This evolution indicates a need for documentation and resources aimed at agents, who can, in turn, relay information to end-users in a more tailored and efficient manner.
As AI models continue to improve, the role of human educators might pivot towards refining the core knowledge that machines cannot yet replicate, thus redefining the educational landscape.
"Want to see our faces? Follow the show on Apple Podcasts, Spotify, or wherever you listen."
The show can be followed on various podcast platforms, which enables listeners to access new episodes weekly.
In addition to listening, viewers are encouraged to sign up for emails to stay updated on show content.
Transcripts for every episode are available at no-briers.com, making it easier for audiences to follow along or revisit discussions.
