• xkcd #1425 (2014): "Check whether the photo is of a bird"
• GIS lookup? A few hours.
• Bird recognition? A research team and five years.
• "In CS, it can be hard to explain the difference between the easy and the virtually impossible."



• And yet, unbeknownst to xkcd, by 2012, someone had already cracked it...

• Hinton: backpropagation pioneer (1986), mentor to Yann LeCun, "a near outcast among his colleagues" for refusing to give up on neural networks
• AlexNet (2012) vs. LeCun's LeNet (1998):
  "at the level of theory, astonishingly little had changed"
  • 10x larger images, 2x larger convolution kernel
  • a few more layers, 1,000 categories instead of 10
• So what changed? The hardware. GPUs made training large networks practical for the first time.
• "AlexNet was no mere contest entry. It was a moment of vindication a quarter century in the making."

• Two Nvidia GPUs, running in parallel, conducting round after round at maximum speed. Hours become days, and days stretch into a week.
• A blacksmith's hammer against glowing steel. One increment at a time, until near-invisible perturbations become mountains and valleys, reaching out into a multithousand-dimensional hyperspace.
• A thousand Dalmatians accumulate here, a thousand laundry hampers there, a thousand marimbas somewhere else... The algorithm hasn't merely 'seen' these things; it's become them.
• This neural network, the largest our field has ever seen, trained by more data than any in history, can generalize.

• Hinton's lifelong bet: intelligence emerges from billions of tiny connections, not hand-coded rules
• Hinton mentioned his 1985 tiny language model: turn words into feature vectors, let features interact to predict the next word, back-propagate errors — the ancestor of every LLM today


• What LLM AI system looks like: 1,000 shadow copies of intelligence traversing solution space simultaneously
• Like Naruto's Shadow Clone Jutsu: each copy is fragile, fades quickly, but the swarm explores every path at once

• Model capability is doubling every 7 months.
  You need to recalibrate every 3 months what the latest model can do.


• Don't over-steer. Don't guardrail heavily.
  Give the model breathing room to innovate.
  • Heavy scaffolding and rigid prompt engineering fight against the scaling law
  • What you hand-tuned for last month's model may hobble next month's


• It's just a while loop

• An advanced civilization built the Deep Thought computer to ask, What is the Answer to Life, Universe and Everything? It returns 42. Nobody understood -- they never knew the real question. So Deep Thought built Earth -- a biological computer -- to figure out what the question actually was.


• A smart agent works the same way: it infers from your prompt, then reaches into the world -- asks clarifying questions, searches docs, reads between the lines
• Your prompt can be imperfect, preliminary. You can't plan everything ahead. Don't Panic, partner with your agent, iterate.

• In the agent era, claude code touches the physical world and runs in self-iterating loops.
• Human need a lot of tools to finish the job too,
  • Humans can't multiply large numbers in their heads — so we use calculators
  • Humans can't picture what a website looks like from CSS/HTML — a browser renders it exactly
  • Humans can't guess how fast C++ runs —
    a profiler measures it scientifically
• Give agents what humans always had: symbolic computation, validators, linters, compilers, unit-test environments, web search, GPUs, robot arms, and sensors. Stand on giants, don't spend tokens rebuilding centuries of human effort.

Andrej Karpathy (@karpathy), Feb 2, 2025:

• "There's a new kind of coding I call vibe coding, where you fully give in to the vibes, embrace exponentials, and forget that the code even exists."
• "I 'Accept All' always, I don't read the diffs anymore."
• "When I get error messages I just copy paste them in with no comment, usually that fixes it."
• "The code grows beyond my usual comprehension... Sometimes the LLMs can't fix a bug so I just work around it or ask for random changes until it goes away."
• "It's not too bad for throwaway weekend projects, but still quite amusing."

• Erik Schluntz, Member of Technical Staff at @anthropic-ai
• You forget the code exists, but not the product exists!
• The tree metaphor:
  • Branch and stem nodes stay fixed — tested, stable APIs and data contracts
  • Iterate crazily on leaf nodes — UI, prompts, config, experiments
• As leaf nodes stabilize, they grow into new stem
• Never vibe code your core infrastructure
• Always vibe code your experiments
• The tree grows outward. Stability propagates inward.

• "Ask what you can do for Claude"
• Do our agents have what they need to succeed?
  • Can it read our doc easily?
  • Are there tribal knowledge locked in people's heads?
  • Why are we doing things so differently vs opensource / big tech?
  • Do we have clean, idiomatic code?


• "You point the thing around and it shoots pellets or sometimes even misfires and then once in a while when you hold it just right a powerful beam of laser erupts and melts your problem."

Dan Shipper (Every Inc): "Each unit of engineering work should make subsequent units easier — not harder."

• Traditional development accumulates technical debt. Every feature adds complexity. The codebase becomes harder to work with over time.
• Compound engineering inverts this. 80% is in planning and review, 20% is in execution:
  • Plan thoroughly before writing code
  • Review to catch issues and capture learnings
  • Codify knowledge so it's reusable
  • Keep quality high so future changes are easy

TLDR: Use Plan Mode interactively. Document. Refactor aggressively. ABSOLUTELY NO QUICK AND DIRTY.

• The most powerful antidote: don't stuff everything into one context. Delegate like human.
• Subagents (spoke): like Despicable Me minions
  • Dispatch a minion with a clean, tiny context
  • It does one job, reports back, and disappears
  • Your main context stays uncluttered
• Agent teams (swarm): like Minority Report precogs
  • Multiple agents work in parallel
  • They email and challenge each other as equals
  • The team scales horizontally
• Delegation is not just a context rot trick.
• It is how you scale intelligence. Just like human.

• A model that can one-shot 99% of the time worth 5 times the model that can one-shot 95%
• Because we're optimizing for human engineering time There is no reason not to use Opus in high stake tasks.
• Every model failure that reaches / slips pass a human is expensive:
  • The human is now the slowest component in the loop. But we must be diligent to catch AI Slops!
  • How to QA at choke points better?
• Answer: Design our tooling to speed up human evaluation. Craft telemetry. Don't dump a wall of text, show green/red diffs that activate the human's biological vision neural network!

• Your agent can't take responsibility. You must.
• Legal liability, field issues, SLA commitments, none of these transfer to your AI
• You still own every mistake your agent makes
  • Deloitte fined for submitting hallucinated references in a government report
  • Papers retracted, reputation tarnished after citing hallucinated papers
• The bar of quality must be high with or without AI
• AI makes you faster. It doesn't make you less accountable.

AWS Reinvent:2025 CTO Werner Vogels: "You are the renaissance engineers with a new set of tools"

• A comforting message. A psychological massage.
• And then he retires.
• And then AWS lay off another 16,000.
• The tool metaphor is incomplete
  • This is not a better hammer
  • AI already achieved super human performance in Go, SWE-bench, Graduate Level Questions, International Mathematic Olympiad
  • And scaling laws continue to hold.
  • We should be humbled and put down our pride, but what is left for us to do?

• Amodei (Anthropic) and Hassabis (DeepMind) at Davos 2026


• Steer it one way: you cut cost and lay people off. Produce AI slops with not enough human supervision.
• Steer it the other way: you innovate, get 10x engineers who build ever more ambitious projects.


• The cost of slow adoption is deadly:
  • Competitors can 10x us
  • Our customers just vibe their own solution

Jenny Wen, Design Lead at Anthropic (ex-Figma Director):

• AI raises the floor. A PM can one-shot prompt a working prototype. Your work has to be better than that. What remains? Craft, Taste, Quality.
• Intuition is not guessing. It is reasoned judgment from deep domain knowledge. "Having great intuition is something to be respected and to aspire to."
• Make something that makes people smile. FigJam's emotes had usability issues during prototyping, but people were smiling and laughing. No problem statement led to it. Just a team that cared deeply.
• "If you could follow a process, anyone could do it." Trust ourselves again: to try new things, to feel and know when we've built something great.

Boris Cherny (Creator of Claude Code, 2025 December):



• In the last thirty days, I landed 259 PRs — 497 commits, 40k lines added, 38k lines removed.
• Every single line was written by Claude Code + Opus 4.5.
• Claude consistently runs for minutes, hours, and days at a time (using Stop hooks).
• Software engineering is changing, and we are entering a new period in coding history.
• And we're still just getting started.

This is the bar of software now. Velocity is this high.

Dario Amodei's essay on what powerful AI could achieve if everyone works together

• The pessimist sees job loss. The optimist asks: what can we accomplish with 50 million Nobel laureates in a datacenter? It could be a time of plenty.
• The Compressed 21st Century: 50-100 years of progress in 5-10 years across five domains
  • Biology/health, neuroscience, economic development, governance, work and meaning
• As humans, we embrace the exponentials:
  • Race to the top, not Race to the bottom
  • Build simple things that work
  • Build not for this model but the next

• Carl Sagan's Contact: when a technological advanced alien civilization observes humanity, they see ants.


• We are now in contact with an alien form of intelligence
• When it can outcode, outplan, outscale us — where is our merit?
• Carl Sagan's answer:
  • Music. Lovingkindness. Dreams. Adaptability.
  
  
• Humanities. So my coping mechanism: Dream big, adapt, and keep building. As a Data Scientist, there has never been a more exciting moment in my life than now. Aren't you curious what we can build with the Swarm?