Captain America, Q, and the Joker: Samuel Colvin on Coding Agents
Pydantic founder Samuel Colvin on Latent Space: coding agents as superheroes and the four rules of 100x LLM coding.
The Episode
Back in March, swyx sat down with Samuel Colvin, founder of Pydantic, on Latent Space. The nominal topic was Monty, Pydantic’s ultrafast Python interpreter for AI agents; the conversation ranged much wider.
I video-snip podcast moments worth keeping and feed them to my knowledge refinery (the Hermes Agent + Obsidian rig from my last post). This episode produced 14 snips, the most so far for a video (video snipping is a recent addition). Colvin is uber-quotable. My top seven:
800 Nanoseconds
Monty’s pitch compressed into one number. The standard answer for agent code execution is a sandbox — Colvin’s answer is an interpreter living inside your process:
The other massive win is latency, because if you can have a Python interpreter that runs inside the same process… in a hot loop, we can go from code to execution result in under a microsecond, like 800 nanoseconds. In reality, it’s single-digit microseconds to run code, or single-digit microseconds to run the next step of a REPL, or single-digit microseconds to call a function on the host. Whereas creating a Daytona sandbox for me was taking one second.
A Pydantic-created Runtime That Doesn’t Allow Pydantic
The tradeoffs, delivered with a founder’s self-awareness:
The biggest downside of Monty is that it is not full CPython. We are implementing it all ourselves… There’s no support for third-party libraries… we’ll never be able to speak the CPython ABI and install Pydantic or install NumPy or something. You can’t yet even define classes in Monty… We have to work out which bits of the standard library we want to manually go and implement. This is some enormous task. And also kind of self-sacrificial… This is a Pydantic runtime that doesn’t allow Pydantic.
The Four Rules of 100x
The meatiest snip of the episode. Colvin set an agent loose on Python’s built-in functions, and it finished in 2 hours what he figures would have taken an experienced developer weeks. Then he names the four conditions that make that possible:
My take is that there are four things where, if you can cover all four of these things, LLMs are not like 3x faster or 5x faster, they’re like 100x faster. Internal implementation is well known to the model: it knows how to implement a heap, implement a bytecode interpreter. The external API is well known to the model: you don’t need to explain how Python should work. It just knows it in its soul, in its weights. Thirdly, unit testing is really fucking simple. You’re just like, does it match Python? And lastly, you don’t have to bikeshed at all about what the interface needs to be… It’s all just defined by Python, so there’s no need for the humans to go and argue. It just works. If you cover all four of those things — that’s why everyone is basically cloning Redis right now in Rust… Tasks get, I would say, ballpark 100x faster.
“It just knows it in its soul, in its weights” belongs on a t-shirt.
Scoring Dario
Pydantic gets GitHub issues from important people, and the team once fumbled one:
It started off because one of the OpenAI co-founders created an issue on Pydantic, and we just closed it and said it was wrong. And so we have this [Chrome extension] that injects itself and tries to summarize someone, and it gives them a brutal score of one to five on how important they are. So when Dario next appears in an issue, we’re not just going to say “that’s dumb” and close it.
Captain America, Q, and the Joker
My favorite snip of the episode. swyx asks what coding agents Colvin used to build Monty:
My take is that you can think of each of them as, if you thought of them as a superhero: Claude Code is like your clichéd Captain America. Mr. Right. Pretty competent, does most things right, a bit overconfident, but whatever, fine. Codex is this neurotic, geeky, the kind of Q guy in Bond, who’s very specific about little things that don’t particularly matter. And then Gemini is like the Joker. It’s just this unhinged lunatic that will occasionally do an incredible job, but half the time just delete all of your files. And I almost never run Gemini with the capacity to do anything other than [read].
Since the episode dates from March, I’ve no doubt that Colvin’s model lineup and characterizations have changed since, but these are too funny to miss. Watch at 16:36
90 Seconds Versus Half an Hour
Why the Joker still gets work: Colvin pipes a branch into Gemini in read-only mode, gets back a review report, trims the bad findings, and hands the rest to a coding agent to implement. The speed math:
The model sits there and churns for a minute or two and returns you a report. Whereas Codex is gonna go through and agentically investigate all of the changes and try and link them up… And maybe it does a marginally better job, but one can take honestly half an hour and one is gonna be done in like 90 seconds. And so quite often that Gemini first review will find things that Claude Code and I have done wrong, to fix more quickly.
And the follow-on, on the industry push toward ever-more agent autonomy:
There are many situations where I would rather it churns for 30 minutes when I could do it in 10, but there are genuinely situations where I’m like, I could have done that change faster.
More Than Memory
The forward-looking one. Ask the model for its answer plus the optimal Python code to run the same task again, feed that code back on the next run, and it gets it right the first time. Then the bigger claim:
I think that there is an enormous opportunity for effectively a new layer of state within applications. You could think of it as memory, but it’s often a lot more than memory. It’s the current state of agent optimization: some of which will be code, some of which will be model choice and model settings, some of which will be system prompt.
The Other Seven
That’s 7 of the 14 snips I took, and the other 7 are good too. Watch the whole episode:
