Lecture by Andrej Karpathy

https://www.youtube.com/watch?v=LCEmiRjPEtQ

1. Three Eras of Software

• Software 1.0 👨‍💻: Traditional code (e.g., C++, Python) written by humans.
• Software 2.0 🧠: Neural networks (model weights) trained on data.
• Software 3.0 🤖: LLMs (ChatGPT etc.), where programs are natural language prompts (English).

"We are now programming computers in English - it's insane!"

2. Is LLM the New OS?

• Analogy with Operating Systems:
  • LLM = CPU 🖥️
  • Context window = RAM
  • Multimodality and tools = APIs
• 1960s of AI: Currently LLMs are expensive and run in the cloud (like mainframes), but will soon be local (like PCs).
• LLM Problems: Hallucinations, "spiky intelligence" (super smart in one area, dumb in another), vulnerabilities.

3. How to Work with LLMs?

• Partial Autonomy:
  • Examples: Cursor (autocode assistant), Perplexity (AI search).
  • "Autonomy slider": From suggestions to a full agent.
  • GUI is critical: Visualizing changes speeds up verification.
• Best Practices:
  • Clear prompts → fewer errors.
  • Keep the AI "on a leash" (too large changes are hard to verify).

4. Vibe Coding — Programming for Everyone

• Now everyone can code just by describing tasks in English.
• Example: Karpathy built an iOS app in a day without knowing Swift. But deployment and DevOps still require manual work (and that's a pain 😅).

5. The Future: Infrastructure for Agents

• Need "Documents for AI":
  • lm.txt instead of robots.txt — instructions for LLMs.
  • Markdown documentation (like Vercel and Stripe).
  • GitHub → Ingest (converts repository to text for LLM). https://gitingest.com/

6. Summary

• We are in the 1960s of AI — everything is just starting.
• Transition from "crutches" to agents will take years (don't believe the 2025 hype).
• Iron Man suit analogy: Currently augmenting humans, but moving towards full autonomy. The future belongs to a hybrid of humans and AI!

Discussion on HN
https://news.ycombinator.com/item?id=44314423

LLM as a new programming paradigm

Arguments "FOR":

• "English — the new programming language": This is a fundamental shift from deterministic, formal languages to probabilistic ones, allowing non-programmers to create software ("vibe coding").
• A new tool: LLM is another tool in the developer's arsenal that complements, rather than replaces, existing approaches.
• Dealing with uncertainty: Programmers have always dealt with non-determinism (API responses, user input), so working with LLMs is just an extension of this practice.

Arguments "AGAINST" (Skepticism):

• Formal languages are an advantage, not a drawback: They provide precision, reliability, and verifiability, which are the foundation of engineering. Abandoning them in favor of natural language is a step back towards "magical thinking."
• LLM non-determinism is dangerous: Unlike an API error, an LLM can produce "garbage that looks like gold" — a plausible but completely incorrect answer that is difficult to detect.
• Hype vs. reality: Many believe that the capabilities of LLMs are greatly exaggerated, comparing the current excitement to the "cryptocurrency bubble."

The developer's role is transforming:

• From writing code line by line to "context wrangling" and prompt engineering.
• The human becomes the verifier and curator — the one who provides a fast "feedback loop" (generation → verification → correction).
• Some fear that this will devalue the profession, turning engineers into "AI QA testers." Others see it as an opportunity for experts from other fields to create their own tools.

Practical tools and challenges

• Structured Outputs: Using JSON mode is a "superpower" that makes LLM output predictable and suitable for programmatic processing. This is often an underestimated tool.
• Determinism vs. Chaos: LLMs are not entirely random. At temperature 0, they are deterministic but "chaotic" (small changes in input can lead to large changes in output).