AI Coding
I’ve now produced a few apps using AI coding assistants. A deep-time world timeline (https://github.com/garyo/deep-time-timeline, check it out at https://deep-timeline.org/), an AI-assisted personal knowledge management app written for my own use (https://github.com/garyo/gco-llm-pkm), a WebGPU image-shader-evolver experiment, a web-based packing list app (https://github.com/garyo/packzen, at https://packzen.org), and a software construction tool called pcons. Claude Code has already improved several fold since I started doing this kind of development six months ago, which itself is noteworthy. I’ve heard that Anthropic is now writing much of Claude Code’s new functionality with claude itself. This really does seem like a tipping point to me.
I’m an end-of-career senior software developer and architect; I’ve been a CTO and individual contributor, I’ve managed developers, product lifecycles, and written big projects in all kinds of languages and toolsets. These thoughts are from my perspective; I imagine others will have very different feelings and ideas sparked by the AI revolution we’re living through. But to me, it feels like one of the most significant changes I’ve lived through, if not the most. This comes with a slew of benefits, but even more caveats and concerns. More on that below.
Speed: Idea to Working App
For me, AI coding assistants like Claude Code have drastically reduced the activation energy of starting a new project. I’ve kept a backlog of “wouldn’t this be cool” ideas for decades; most of them are just too much work for a very uncertain business model, not knowing if they’d even work or not. Or I see some piece of software out in the world and think “I know I could do that better; I could make something that works just the way I want” — but is it worth spinning up a whole project for a few weeks or even months of work, just to make something a tiny bit more customized to my taste?
The answer, with Claude Code, is emphatically yes. I can write a spec, have Claude code up an initial version in an afternoon, and spend a few more days adding features, trying different approaches, even starting again, and in a long weekend or a week or two have something really quite decent. I’m pretty happy with packzen at this point.
Finding meaning
I don’t think I was ever a mythical “10x” engineer, but I’ve always been pretty good at it, and I’ve been proud of the work I’ve done, both the user-facing results and the code quality itself. So what does it mean for me to turn over a large chunk of that process to an algorithm?
A lot of people, myself included, are very concerned about how anyone will learn programming if we leave all the grunt work to machines. I personally am happy to direct and collaborate, more like a conductor than an orchestra musician, but without musicians, a conductor is nothing. Where will those people come from in the future? Will new developers learn high-level skills (project management, UX design, the tradeoffs between time and space, performance and maintainability, and the importance of good API design, knowing just how much flexibility to design in, where to draw the lines in testing, and so on)? I learned those skills through a lifetime of programming in all kinds of languages, from the lowest to the highest levels, creating systems large and small, and thinking deeply about what worked and what didn’t. Can those skills be taught by prompting an LLM? It may be that they can, but right now I’m skeptical. It’s all too easy to create “slop” that satisfies no one.
I note that this isn’t just an issue with software developers. Marketing, legal, HR, governance, medical, and customer support roles are all at risk of being replaced by AI (in some cases they’ll likely do it better than humans, and in some cases not — and probably nobody in power will care). If we don’t have a way for people to learn their jobs by doing, then where will future experts come from?
But for me personally, at this point in my life, I’m less interested in learning the details of the Clerk auth API or how to set up state management in Astro, or even puzzling through a C++ metaprogramming task, than I am in getting my product idea turned into something I can see and play with and show others. I want to make helpful and interesting things. Frameworks will churn, as they do; languages will evolve, dependencies will go stale, and things will have to be adjusted. For me, that’s no longer the fun part. I’m glad to turn that over to an always-cheerful always-helpful assistant.
What does the act of programming in a text editor or IDE mean?
One of the things I’m already feeling myself losing, doing development this way, is the sense of deep flow a programmer gets into when solving a big problem. You’re holding so much in your head at once, you no longer feel time passing, and it’s like the code is flowing out of you. You mind-meld with your editor or IDE. You’re building a crystalline structure out of literally nothing. That kind of experience is special, and prompting an LLM to create code is qualitatively different. It does feel more like a conversation, with all the give and take that entails, suggestions and corrections going both ways, so it’s fun in a different way. But it’s not the same as diving into the deep end of a problem knowing you’re going to succeed but having no idea how, wrestling with the minutiae and syntax errors but also the big ideas, questions of data representation and optimization and generality. I think people will miss this in an LLM-driven future. Maybe we’ll all become artists, musicians and dancers to find that kind of flow, but I doubt it.
Learning vs. getting results
Programming has always been a learn-by-doing experience. Sure, you can (and should!) take classes in data structures and algorithms, database design and compiler architecture, but just like playing an instrument, you don’t really get it til you get it in your fingers. It may be that kids will collaborate with LLMs and learn a different set of skills which may prove just as useful, but at this point the jury is very much out on that. There’s a gap between what claude can do and what a full system design needs, a gap that has to be filled by human intelligence and judgment. If LLMs get better, they may be able to fill that gap. But today, it has an awkward shape because the skills needed to hone that human judgment are going to atrophy as the old greybeards like me age out, and the new devs haven’t built the instincts and experience needed. It’s going to be a very interesting decade or so.
Collaboration, not assistance
Coding with an AI tool like Claude (I use Opus 4.5 today) is starting to feel much more like working with another person than it did even a few months ago. It asks questions, it suggests ideas. I often start with a vague outline of my vision, with a few important signposts, and say “suggest three ways to approach this, with analyses for each one, and let’s discuss.” Things get deep really fast, and between us we can flesh out a detailed design that to me feels right. And if it turns out it has blind alleys, there’s no ego involvement — Claude is happy to backtrack and take a different approach. And it’s really good at refactoring.
The importance of tossing ideas around
I’ve always been at my best as a software engineer when I’ve had someone to bounce ideas off of, someone who can say “what about this?” or just listen. Through my career at GenArts, Karl and I had this running joke, where one would say “Can I explain this code to you so I’ll see where the bug is?” because often the other person doesn’t even need to say anything. You get to the point where you say “…and then we copy to the heap and… oh yeah. That bit never gets freed.” This kind of collaboration and understanding makes software development more productive, but also a lot more fun.
Tools like Claude Code are now getting to the point where discussions like that feel fruitful and engaging and even sometimes inspiring. I’ll say “let’s think through options for the UX for this dialog; I want the autocompletion to work like this, but it needs to be clear that they already have that item in the packing list” and Claude will suggest some options, I’ll say “That’s pretty good, but it’ll take a lot of space, right?” “But we could have a top section, small gray font, with an indicator, that doesn’t take focus and keeps the user in their flow.” Yep, that is awesome. Let’s do it like that.
Make no mistake, this is deeply weird. How much of programming is really a social activity? Nerds are famously introverted, but we know that we’re writing code for other humans (even “future us”) as much as for compilers and the machine itself. How much can a virtual collaborator replace that human contact? Should it? What kind of future are we creating for ourselves?
Is a coding agent the new compiler?
I rarely look at the generated assembly or bytecode from a compiler — only when I’m truly interested in bare-metal performance. Most of the time, I trust the tool to generate reasonable assembly code. I wonder if, in a few years, this is how we’ll treat coding assistants? We’ll tell them what to do, review the high level results, and trust that the code they wrote accurately represents our intent. I’ve seen articles on the net saying it’s not the same, because writing code and compiling it is deterministic; there’s only one correct executable program that represents a certain source file. But for at least a decade or two, since the advent of branch prediction and speculative execution hardware, that’s no longer even close to true. Modern compilers have incredible optimization engines that create tighter machine code than a human ever could, and we’ve long ago turned over those fine-grained decisions to them. They loop-unroll, auto-parallelize, and reorder code while maintaining correctness. It’s absolutely true that a text-based description of a program for an LLM is a lot looser than C++ source, but I think this will eventually be seen as a difference of degree, not kind.
But is it thinking?
Here’s a quick anecdote; I’ll try to avoid anthropomorphizing, but it gets pretty hard. I prompted Claude Code to add a significant feature to my personal knowledge manager app. It produced a two-page detailed plan with a good UX, careful, secure back end API and good tests. I prompted it to go ahead with the implementation. It wrote a few hundred lines of code and then wrote “Ah, now I understand the problem better. I can see that my approach won’t work well, and it duplicates code from another part of the app. Let me think about a better approach.” It then goes off, writes 250 lines of very nice code, and produces an improved plan. I approve that and it then proceeds to write the whole feature, which works first time.
So what am I to make of this “now I understand” and “let me think”? Does it understand anything at all? Isn’t it just a “stochastic parrot” as some people claim? If a person made those statements, I would have no trouble believing that they do understand the problem better after digging into it. There’s no denying that Claude explored an approach for a while, “decided” that it was going down a blind alley, and was able to pop out and create a better solution. I really don’t see any reason not to believe that it does understand the code, its own approach, and the overall goal. Some people may say it’s not real understanding or real thought unless the system producing the results is “alive” or made of the right organic substrate or has the right quantum synaptic structure, so perhaps I’m just an operationalist in this case, but I think it’s rapidly becoming a distinction without a difference.
I plan to write another essay digging into the philosophical implications of this, but for now I’ll just side with those who think this is a real form of understanding and thought, and probably true agency. Consciousness is another matter entirely though. As a side note from a child of a philosopher, it amazes me that we as a culture are now engaged in practical philosophy — the oldest existential questions are suddenly made manifest and testable, and they affect our daily lives.
What does this mean for the future of development?
Individualized software
I think it’s very likely that within a few years, average people, not highly technical, will be able to tell some system that they want a piece of software (call it an agent, or an app, or an assistant) to do something for them, and the system will write it, debug it, and spin it up for them in the cloud or on their device. They’ll be able to tell it what they like and how to do it better, and it’ll learn their preferences (I’m old, so I like large touch targets; in my younger days I wanted to fit as much info on one screen as possible, to give me all the context) and write the tool just for them. I can imagine a future where Word and Excel and the paint apps and video editing tools we know today give way to full-custom (or maybe 50% custom with open-source infrastructure) bespoke apps written just for one user, and the apps themselves evolve as they’re used, maybe sharing info with other users.
Self-improving tools
I’ve already set up my AI personal knowledge management tool to self-improve; it learns what queries work, what I like and don’t, and can write shell scripts and improve its own prompts. It reviews our chat logs and looks for ways to do better. Why can’t all software behave like this, once we have software that writes software? I think that’s coming. Software will truly adapt to the ways we like to work and play. It’ll automate the drudgery and surface interesting ideas. It will also be able to find ways to subtly manipulate us and shape our preferences and goals, like social media on steroids. Do we use our tools, or do the tools use us? In a world like that, I’m not even sure it’ll be obvious who benefits — those manipulations may not even be human- or corporate-directed the way social media is today.
Bigger Picture
What does all this mean for the future of society and culture?
This isn’t the place for a big discussion of what AI in general will mean for society and culture, or whether machines can be conscious, and in any case nobody has a clue how this will all play out. But it’s clear to me that we’re at a tipping point, that we’ve for the first time in human history created tools that think, that have agency, and can produce new ideas on their own. It is going to create huge disruptions to the future of work, certainly. The effects on art and culture are more uncertain; plagiarism is a huge concern, and sadly AI has already started to expose how much of our writing and music and imagery are easily automated to the point where people can’t tell what’s AI-generated and what’s created by humans with feelings and desires. I think there are very real dangers to governance (deepfakes and propaganda), law, and medicine. The compute requirements are, at least today, incredible. That may very well change as we discover new cheaper ways to perform inference and train models, and find new kinds of more efficient world models. It took humanity a long time to run a 4 minute mile, but once it was done once, suddenly more people could do it. I think now that we’ve seen real “thinking machines” we’ll see a burst of creativity and innovation in that space, not to mention recursive self-improvement by the models themselves. Will these machines outstrip us, replace us, care about us? Will we be able to turn them off? (Sure, you can exist without a smartphone and credit cards. But nobody really does.) It’s going to be an interesting decade.
Risks
I plan to write another essay on the risks of AI as we leap off this cliff, perhaps the largest social experiment we’ve ever constructed. For now, I’ll leave you with this incomplete short list:
- Criminal use: deepfakes, agentic hacking, identity theft, data kidnapping
- Political misuse: deepfakes, disinformation, misinformation, election interference, mass surveillance
- Scientific/military misuse: chem/bioweapon design, autonomous drone weapons, terrorism
- Flooding channels with slop: music and art, discourse, law
- Emotional dependence, lack of human contact, addiction
- Skewed harm/benefit ratios magnifying societal inequalities
- What I call “epistemic shock:” the knowledge that we’re not the most intelligent species in the universe anymore
- Job losses as the economy readjusts or fails to readjust (law, marketing, retail, customer service, logistics, etc.)
- Misalignment causing trauma, suicides etc.
- Deceptive AI behaviors combined with self-protection: check out the 1970 film Colossus: The Forbin Project
- Runaway self-improvement: could cause divergence from human goals/needs, disengagement, or worse
- Energy use (I don’t think this will end up being that serious as AI finds better, more efficient algorithms to improve itself)
