Andrej Karpathy: Software Is Changing (Again)
Y Combinator39:312025-06-19English
Chapters
In this talk, Andrej Karpathy calls large language models programmed in natural language Software 3.0. He recommends starting with products that let people review results quickly and gradually hand over more control, rather than aiming for fully autonomous agents from the start.
Overview
Andrej Karpathy: Software Is Changing (Again)
- Karpathy distinguishes Software 1.0, 2.0, and 3.0 and uses his experience of neural networks gradually taking over C++ functions in Tesla Autopilot to show how a new paradigm changes the software stack.
- He compares three analogies: utilities, chip fabs, and operating systems. Today’s shared cloud access resembles early time-sharing, but consumers adopting the technology before institutions is unlike what happened with many earlier technologies.
- He describes models as stochastic simulations of people: they have broad knowledge, but they also hallucinate, make basic mistakes, lack persistent memory, and face risks such as prompt injection.
- Using Cursor, Perplexity, course design, and self-driving cars, he shows how dedicated interfaces, reviewable intermediate outputs, and adjustable autonomy let people stay in control of what they verify and how quickly they hand over control.
- Drawing on how he built an iOS app without knowing Swift and how MenuGen was quick to prototype but slow to launch, he distinguishes the lower barrier to entry offered by vibe coding from the operational hassles that remain.
- He treats agents as new users of digital infrastructure, argues for readable docs and operations agents can call directly, and closes by talking about gradually increasing product autonomy.
Key points
- Software 1.0 is explicit code written by people. Software 2.0 is neural network weights produced using data and an optimizer. Software 3.0 uses natural-language prompts to program large language models. Each has its trade-offs, and developers should be able to switch between them depending on the task.
- Large language models are like utilities that charge based on usage, and also like capital-intensive chip fabs. Karpathy thinks the operating system analogy fits best: the model is like a CPU, the context window is like memory, and the model coordinates tools, memory, and computation to solve problems.
- Model capabilities are uneven: a model can outperform people on some tasks, yet still make things up and commit basic errors. It also doesn’t naturally build up experience in day-to-day work the way a new colleague would, so applications need to actively manage its working context.
- Partially autonomous products should keep manual controls available, show reviewable results in a dedicated interface, and let users adjust the level of autonomy by task. Cursor’s code diffs and Perplexity’s source citations are both designed to help users verify results.
- Fast generation doesn’t mean fast completion: a model can produce thousands of lines of changes in an instant, but someone still has to check that they’re correct. Karpathy prefers to use specific prompts to break tasks into small steps that can each be checked and approved quickly.
- Natural language lowers the barrier to building prototypes but doesn’t eliminate the hassle of going live. The MenuGen demo was running in a few hours, but authentication, payments, domain setup, and deployment took about another week.
- Designing for agents isn’t just about converting documentation to Markdown. Steps that depend on someone clicking through an interface also need to become operations that agents can call directly. The talk gives the examples of curl commands in Vercel’s documentation and Anthropic’s Model Context Protocol.
Chapter notes
From writing code by hand to programming in natural language
- Andrej Karpathy thinks software saw few fundamental changes over roughly 70 years, then went through about two rapid shifts in the last few years. As a result, a lot of software needs to be written or rewritten.
- Software 1.0 consists of instructions people write directly for computers. Software 2.0 consists of neural network weights that people don’t write directly. Instead, they adjust the dataset and an optimizer generates the parameters. Hugging Face is like GitHub for Software 2.0: fine-tuning a LoRA on Flux is like making a Git commit in that space and getting a different image generator.
- In the past, neural networks mostly performed fixed functions, such as mapping an image to a category. Large language models make it possible to program neural networks with prompts, a paradigm Karpathy calls Software 3.0. For sentiment classification, the three approaches are to write Python, train a neural network, or write an English prompt for a large language model with a few examples.
- The Tesla Autopilot software stack turns vehicle inputs into steering and acceleration commands. Early on, it consisted mainly of a lot of C++ code and neural networks for image recognition. As the neural networks grew in size and capability, functions such as combining image information across cameras and over time moved from C++ into the networks, allowing the team to delete a lot of the original code.
- Karpathy thinks the new paradigm is taking over parts of the stack just as Software 2.0 once did. Each of the three paradigms has pros and cons. People working in the field should be comfortable with all three and be able to choose between writing explicit code, training a neural network, and prompting a large language model for a specific function.
What kind of infrastructure are large language models like?
- Large language models are like utilities: labs invest capital to train them, then pay to operate them and charge for API usage, for example by the million tokens. Users expect low latency, high availability, and consistent quality. OpenRouter is like a power transfer switch that lets users switch between models. When advanced models go down, some people can’t keep working, as if they were experiencing an “intelligence brownout.”
- Large language models are also like chip fabs: training takes enormous capital, the technology advances quickly, and R&D secrets are concentrated in a few labs. But software is easier to change, so the barriers aren’t as durable. Using Nvidia GPUs and focusing only on software is like the fabless model, while Google designing its own hardware and training on TPUs is more like Intel owning its own fabs.
- Karpathy thinks operating systems are the closest analogy: an LLM software ecosystem is taking shape around features like tool calling and multimodal capabilities, rather than models becoming interchangeable commodities like electricity. A few closed-source providers are like Windows and macOS, while the Llama ecosystem might develop into an option like Linux. In this new computer, the model is like the CPU and the context window is like memory.
- The model coordinates memory and computing power to solve problems. Just as VS Code can run on Windows, Linux, or Mac, Cursor can switch between GPT, Claude, and Gemini through a drop-down menu. Interacting with ChatGPT directly through text is like using an operating system through a terminal: some applications have graphical interfaces, but there isn’t yet a general-purpose graphical interface that covers all kinds of tasks.
- Things today are like computing in the 1960s: compute is expensive, models are centralized in the cloud, and users connect over the network through thin clients. Each user’s request is just one item in a batch processed in the cloud. Personal computing hasn’t become widespread yet because it still doesn’t make economic sense. However, inference for a single request is mainly limited by memory, and Mac minis are already suitable for running some models, which may be an early sign of local computing.
- LLMs have reversed the adoption pattern of many new technologies. Electricity, computing, and GPS were new and expensive at first, so governments and companies usually adopted them before consumers did. This time, ordinary people started using them first, while companies and governments have lagged behind. Early computers were used for ballistics calculations, whereas Karpathy uses an LLM to ask how to boil eggs.
Models’ exceptional abilities and cognitive shortcomings
- Karpathy sees large language models (LLMs) as stochastic simulations of people. An autoregressive Transformer processes tokens one at a time, using roughly the same amount of compute for each token. The model fits its weights to vast amounts of human-written text from the internet, and some human-like psychological traits emerge as a result.
- An LLM has read far more text than any one person, giving it encyclopedic knowledge and recall. It can even easily memorize SHA hashes. Karpathy compares it to Rain Man: Dustin Hoffman’s character reads the phone book once and remembers the names and numbers in it.
- LLMs hallucinate, make things up, and can’t reliably judge what they know. This has improved, but it’s still not perfect. Their abilities are uneven: they outperform humans at solving some kinds of problems, yet may insist that 9.11 is greater than 9.9 or that “strawberry” has only two r’s.
- LLMs are like people with anterograde amnesia. A new colleague gradually gets to know the organization through work and consolidates knowledge during sleep, building up experience. Models don’t do that naturally. The context window is more like working memory and has to be managed explicitly. Karpathy uses Memento and 50 First Dates to illustrate what it’s like when the weights stay the same but the context is repeatedly wiped.
- LLMs tend to trust their inputs too readily, which leaves them open to security risks such as prompt injection and can lead to leaks of user data.
Partial autonomy and fast human review
- Cursor keeps an editor that users can operate entirely by hand, while also letting models work on blocks of code. Different models work together to handle file embeddings, chat, and the application of code diffs. A red-and-green diff view shows additions and deletions directly, so users can accept or reject them with Command Y or Command N instead of checking each one through chat.
- An “autonomy slider” lets users decide how much control to hand over for each task. Cursor can do basic Tab completion, use Command K to edit selected code, use Command L to edit an entire file, or use Command I to have an agent work across the whole repository. Perplexity likewise offers levels ranging from a quick search to deep research that returns results about ten minutes later, and lists its sources so users can check them.
- AI can generate thousands of lines of code changes in an instant, but people still have to check whether the changes meet the requirements or introduce bugs or security issues. Fast generation doesn’t mean the work moves forward quickly. Breaking tasks into small, specific pieces and writing clear prompts can cut down on rework when the results don’t match expectations.
- Instead of simply telling ChatGPT to “teach me physics,” it’s better to split teaching into two apps: one for teachers to create courses and another to guide students through them. The course is a reviewable intermediate output. Teachers can check the quality and consistency of the content, and the AI is constrained by the syllabus and the order of the projects.
- In 2013, he took a roughly 30-minute self-driving ride in Palo Alto with zero human intervention and thought the technology would mature quickly. Twelve years later, self-driving still hasn’t been fully solved, and there’s still quite a bit of remote operation and human involvement. The same goes for software agents: one perfect demo doesn’t mean reliable real-world deployment. That’s why he prefers to call this the “decade of agents” rather than the “year of agents.”
- At this stage, it’s more worthwhile to build an Iron Man suit that a person controls than to chase flashy demos of fully autonomous robots. Dedicated interfaces can shorten the loop between AI generation and human review, while an autonomy slider lets the product take on more work as its capabilities mature.
Natural language makes prototyping easier, but launching is still hard
- New ways of programming let software act on its own and let people program it in natural languages such as English. Karpathy thinks this means people can start making software without first spending five to ten years learning, as they used to.
- Thomas Wolf of Hugging Face shared a video of kids vibe coding. Karpathy thinks this can be a way into learning software development. It’s also handy for throwing together a small tool over the weekend that you can’t find on the market and that meets your own needs.
- Karpathy doesn’t know how to program in Swift, but he built a simple iOS app in just one day and could run it on his own phone that same day, without first spending a few days learning Swift.
- Because he often couldn’t tell what dishes were from their names on restaurant menus, he built MenuGen. Users take a picture of the menu at menugen.app, and the app generates images of the dishes. Each registered user gets $5 in free credit, and the app is currently losing him quite a bit of money.
- The code turned out to be the easiest part of MenuGen. A demo was running on his laptop in a few hours, but adding authentication and payments, setting up the domain, and deploying to Vercel took another week. What took time wasn’t writing code but clicking around in a browser to finish all the operational setup.
- Adding Google Login is one example. Clerk’s integration docs list a lot of manual steps, telling him to open specific URLs, click drop-down menus, and select options. He asks: if the computer can tell him what to do step by step, why can’t it just do it for him?
Reworking documentation and interfaces for agents
- In the past, people used software through GUIs and programs called it through APIs. Now agents are a new kind of user that reads and acts on digital information. Websites can follow the example of robots.txt and offer an llms.txt file in Markdown that tells an LLM directly what the site does, avoiding error-prone HTML parsing.
- Vercel and Stripe have started providing Markdown documentation that’s easier for LLMs to understand. Karpathy pasted all of Manim’s documentation into an LLM and described the effect he wanted. The model wrote an animation that met his requirements without him having to read through the docs himself.
- Converting the docs to Markdown isn’t enough. The steps also need to be in a form agents can execute directly. Vercel replaces “click” instructions in its docs with equivalent curl commands. Anthropic’s Model Context Protocol offers another way to interact directly with agents.
- Switching a GitHub repository URL to GitIngest combines the files into a single text document with the directory structure, ready to pass directly to an LLM. DeepWiki goes a step further: Devin analyzes the repository and generates dedicated documentation pages rather than just organizing the raw files.
- LLMs can already operate interfaces and click buttons, but this is still more expensive and harder to use, so it’s worth making information and operations easier to access directly. A lot of long-tail software that isn’t actively maintained won’t be adapted specifically for agents, so tools that can operate existing human-facing interfaces will still have a role.
- LLMs make mistakes, so it’s important both to adapt infrastructure and write code directly for agents, and to build partially autonomous products through rapid iteration. Returning to the Iron Man suit analogy at the end, Karpathy expects products to become gradually more autonomous over roughly the next decade.
What it answers
- What’s the difference between Software 1.0, 2.0, and 3.0?
- Using sentiment classification as an example, 1.0 means writing Python code, 2.0 means training a neural network to produce weights, and 3.0 means writing a prompt for a general-purpose language model, for instance by giving it a few classification examples.
- Why compare the LLM ecosystem to early computing?
- Compute is still expensive, models are mostly centralized in the cloud, and users share computing resources over a network, much like early time-sharing. Talking directly to a model through text is also like using a terminal, and a general-purpose graphical interface hasn’t taken shape yet.
- Why doesn’t one successful demo prove that full autonomy is just around the corner?
- In 2013, Karpathy took a roughly 30-minute self-driving ride with zero human intervention, but he says that 12 years later there’s still remote operation and human involvement. From this, he concludes that agents are more likely to evolve over a decade than to see a breakthrough in a single year.
- How can we speed up the loop between AI generation and human review?
- Use a visual interface so people can quickly see what changed, and limit generation to small, specific tasks that are easy to check. That way, the model doesn’t deliver more at once than a person can review efficiently.
- How can existing services become easier for agents to use?
- Use llms.txt or Markdown docs to provide clear information about the site, and offer directly callable alternatives to steps that used to require clicking through a web page.
Quotes
And basically, your prompts are now programs that program the LLM.
It is in our interest to make this loop go as fast as possible so we're getting a lot of work done.
It's less like building flashy demos of autonomous agents and more building partial autonomy products.
Mentioned
- Andrej Karpathy — The speaker and former Tesla AI director, who introduces Software 3.0 in the talk and draws on his own experience to explain how to design partially autonomous products.
- Hugging Face — Karpathy compares it to GitHub in the world of Software 2.0.
- Tesla Autopilot — The talk uses it to show how neural networks took over functions previously implemented in C++, and as an example of how to show users what the system sees.
- 软件 3.0 — Karpathy uses this name for the new paradigm in which natural-language prompts are programs and large language models are programmable computers.
- Cursor — Used to show how a coding app can integrate model calls, display code diffs, and let users hand over control in stages, from autocomplete to tasks that span the entire repository.
- Perplexity — Used to show how search and research apps can support verification through source citations and offer different levels of autonomy, from quick searches to deep research.
- 自主程度滑块 — Karpathy uses it to describe a product design that lets users adjust how much control AI has for each task, rather than choosing between fully manual and fully automatic operation.
- Waymo — Through a friend who worked at Waymo, Karpathy got to try an early self-driving system, which led him to reflect on how long it takes to go from a demo to reliable autonomy.
- 氛围编程 — The talk uses this term for getting a model to write a software prototype quickly using natural language, and treats it as a way into learning software development.
- MenuGen — The app Karpathy built with help from natural language to illustrate menus, and an example of the contrast between how long it takes to build a prototype and how long it takes to launch it.
- llms.txt — A text file proposed in the talk that uses simple Markdown to tell large language models what a site contains.
- Model Context Protocol — The talk presents this Anthropic protocol as one way for software infrastructure to interact directly with agents.