AI Experimentation, or Doctrine?

Cross-posted on Substack.

One of the disorienting aspects of the new AI world we’re hurtling into is that we don’t know where on the terrain we are.  Are we at the threshold (or the “foothills,” as Google DeepMind’s Demis Hassabis put it)?  Are we well into the interior?  Or if we think of AI development as the slope of a “maturity curve,” are we at a gentle beginning or in the middle of a steep climb? 

It’s hard not to feel like everyone is higher up the mountain, or further along the slope, than you are.  I work in the legal industry, and just months after ChatGPT burst onto the scene, the legal world quickly reacted with the concern that “clients are expecting us to use this.”  How to use it, no one knew.  Should it write my emails?  Can it write my legal marketing materials?  Now, law firms are building their own half-billion dollar AI tools, forcing others to feel like they need to catch up.  Even the major AI players are feeling left behind.

But if we’re really at the threshold, or on the foothills—I happen to think we probably are—what are we in the process of building?  What do we think we even know right now?

Spend some time in the AI social media ecosystem. It’s illuminating to see how tech companies in the middle of the AI boom (or bubble) are talking about using these tools.  On June 12, 2026, AWS posted a Ted-talk-esque YouTube presentation entitled “A leader’s guide to advanced team structures in an agentic world.”  Watching the video is well worth half an hour of your time. The talk is a peek into how one of the most profitable companies at the heart of AI technology is thinking about organizational structure and leadership.  But note the title, and note the tone of the talk:  this is presented as best practice, the doctrine of "how-to-think-now." 

This sort of thing is everywhere.  Y Combinator, the San Francisco (formerly Boston) tech incubator where Sam Altman got his start, posts fascinating videos all the time from entrepreneurs; many of them are now focused on how they’re using AI—and how you should, too!  An example that sometimes wakes me up in the middle of the night in the same way thoughts of eternity used to wake me up is of this entrepreneur talking about agents monitoring agents, and having access to every nook and cranny of data accessible to the company. The talk could simply be read as “here’s something really fascinating that some companies are doing with AI.”  But it was instead couched as “here are the best practices you have to follow or you will be left behind.”

I find this all mystifying.  What best practices?  Three years ago we were talking about best practices in having AI write your emails.  Now we’ve quickly realized that deskilling and cognitive outsourcing are real problems:  the more you have AI do your thinking for you, the less you think.  As one BCG study put it, “When Everyone Uses AI, Companies Risk Losing Critical Skills.”  Three years ago, educators seemed excited about using AI in classrooms.  Now?  There are concerns about widespread cheating and loss of learning.  And back to law firms:  as clients (and firms) expect lawyers to have first drafts of work done by robots and not juniors, how will we train the lawyers of the future?

Don’t get me wrong:  I’m not being a doomer.  Personally, I have a love-hate relationship with AI; it’s a fascinating technology with a deep and interesting history, and the tools themselves can augment work and thinking.  (I recommend Karen Hao's highly readable Empire of AI if you want to find an entry point.) I also have to know how to use these tools because I work as a data and technology lawyer:  to effectively do my job, I fundamentally need to understand the tools that my clients are using, sometimes better than they do.  While my preference would be to live in a cabin in Vermont surrounded by old analogue books and a single rotary phone at my disposal, talking about Plato and hiking the Green Mountains, I can accept that that’s not the world I currently live in.

Instead, I’m doing what everyone else is doing:  I’m experimenting.  I’m playing with the technology as it is today. I'm trying to understand agentic workflows by trying to create them. I'm figuring out what it means to vibecode. I'm building apps and querying large data sets. That's very different from what I was doing two years ago, which might have consisted in writing about David Ortiz's last plate appearance in the style of a Star Wars crawl. I'll be continuing to experiment, because what the technology is today may be very different from what the technology is tomorrow.  Just a few years ago, the state of the art was a “blurry jpeg of the web,” a chatbot you could ask questions to that would spit out often-incorrect but persuasive-sounding information.  Today?  RAG, MCP, skills, agents, open weights, and all on the road to AGI or even ASI.  Do most people even know what these things mean?  What they can do?  The truth is, the vast majority of us have no idea just how powerful that Claude app is that waits in the phone in your pocket.  If you’re using Generative AI as a super-query tool, you’re only scratching the surface of what it can do.

So why are we creating doctrines around this tech?  How can there possibly be best practices?  Why is anyone providing any practical “oughts”?  To my mind, it’s not honest.  It’s either marketing, FOMO, or simply a misguided assumption that the tech of tomorrow is going to look like the tech of today.  We can be confident it will not. But we should have no confidence about what it will look like – most of us barely know how to fully use the LLMs that reside on our phones.

The current tech moment is a fundamentally optimistic one.  It rhymes with a certain kind of optimism inherent in capitalism itself:  the idea that human progress is linear, that technology leads to prosperity, that we all benefit from the efforts of those who innovate and seek profit from their innovations.  You can find this optimism in recent polemics, like Marc Andreessen’s “The Techno-Optimist Manifesto,” or Ezra Klein and Derek Thompson’s book, Abundance—each of these adopting a version of this optimism from wildly different policy stances.  I don’t cite these works to uncritically accept their arguments, but instead to observe that there is a powerful kind of ethos that serves to morally justify the AI world we’ve entered. That ethos takes as a given both that AI development is being done for humanity’s benefit, and that humanity will in fact broadly benefit.

There are strong reasons for skepticism, but I think this optimism is both sincere and part of the reason that so many are ready to bottle, package, and codify the current moment into a set of principles for the new world we’re living in.  Look at what AI can do!  It’s going to fundamentally change everything, hop on the wagon and let’s get started! 

But this enthusiasm to create AI dogmas is fundamentally misguided.  We are not living in a moment of consolidation and codification.  We are in a moment of rapid change, the end of which no one can predict—least of all those actually creating the tools that are ushering in that change.  It would be foolish to take a snapshot of rapids and assume we have determined the contours of the river.  Yet that appears to be what many want to do with AI.

Our collective ethos in using AI should be to invite information sharing, experimentation, and study.  It is helpful to see daily new scholarship taking stock of the mental and social effects of AI.  It is useful and exciting to see how cutting-edge, creative entrepreneurs are using AI.  That collective experimentation and analysis is going to continue because the technology is going to evolve in ways we cannot imagine, especially if we take seriously the frontier labs' promise – or threat – that AI will be recursively improving itself.  And that collective experimentation is occurring within and across multiple industries and realms—healthcare, law, finance, education, computing, shipping, government—where the risks, rewards, and spectrum of talent and interests differ. 

There is also a certain policy tension here, which should not be lost on anyone. There is a real need to manage AI risks and harms.  Our repeated policy default position is to let innovators innovate; but that is precisely why we have a paucity of useful regulation, even though having a serious debate and deliberation about regulation can help us all be thoughtful about the AI we are creating, deploying, and using.  "Let innovators innovate" is also an easy and well-worn excuse that allows those who profit the most to escape liability for harms that come from their untested technologies.  The fact that we don’t really know where we are on the map may be a legitimate reason to take care before we regulate too prescriptively. But it is strange to watch an industry draw borders around how organizations should structure themselves and use the technology, on the one hand, and push back on regulation, on the other.

We need to take seriously what the technology can do today and what it can foreseeably do tomorrow, both in our personal and organizational uses and in our policy choices. At the same time, we need real humility in knowing that these uses and regulations will need to change, and will almost certainly need to change quickly.  It makes more sense to regulate against harm and leave the creation of practice doctrine to another day than it does to eschew regulation and ossify practice.

Whether we’re at the foothills or near the summit is anyone’s guess.  What is clear is that we are all still learning while the technology is in the middle of development. Let's not pretend that anyone can write the AI textbook just yet.