Some people have integrated Gen AI tools into how they work so thoroughly that they’ve genuinely changed what they can produce in a day. They’re not just using ChatGPT to clean up an email. They’re using Gen AI to draft specs, stress-test logic, synthesise research, generate test cases, scaffold code. They’ve developed instincts for when to trust the output and when to push back on it. They’re well into their 10,000 hours.

Other people on the same team, at the same level, with the same access to the same tools, haven’t. They’re not opposed to AI. They’ve tried it. They just haven’t crossed the threshold where it changes how they work rather than adding another tab to their browser.

It doesn’t map to seniority. Some of the most AI-fluent people on a team are mid-level. Some of the least fluent are senior leaders. The gap isn’t about intelligence. Perhaps the closest I can get is that it’s about curiosity, comfort with iteration, and the willingness to be a beginner again, to feel clumsy at something new.

As a leader, this is genuinely hard to navigate.

You can’t mandate fluency. I’ve seen teams try — “everyone must use Copilot” or “all first drafts should go through Claude” — and it mostly produces compliance without capability. People go through the motions without building the instinct for when AI helps and when it gets in the way.

What I’ve found works better is making the workflow visible. When someone on the team uses AI to do something faster or better, get them to show how. Not in a formal training session. In the actual work. In a standup. In a PR review. Make the craft legible rather than mysterious.

The other thing that helps is giving people permission to be bad at it. The reason some people are fluent is that they spent weeks producing mediocre AI-assisted output before they got good. They experimented. They ‘wasted’ time. They learned what doesn’t work. If your culture penalises visible inefficiency, you’re selecting against the very behaviour that builds fluency.

This isn’t going away. The gap between AI-fluent and AI-adjacent team members is going to widen before it narrows. And the teams that figure out how to close it without creating resentment or mandates are going to have a genuine edge.

I’d love to hear from both sides. If you’ve become fluent, what got you over the hump? And if you haven’t yet, what’s actually in the way for you personally and how could your co-workers best support you?

A few years ago, a small team could move fast but was fundamentally constrained by headcount. You could prioritise ruthlessly, but there was a hard ceiling on how much you could actually produce.

Gen AI tooling has lifted that ceiling. Not removed it (let’s not get carried away!) but meaningfully lifted it. A small team with the right tools can now produce research, analysis, first drafts, prototypes, and working code at a pace that simply wasn’t possible, even eighteen months ago.

Think about what a typical product development cycle looks like at a small company. You’ve got a product person writing specs, a designer exploring options, engineers building, someone writing tests, someone else drafting documentation. Every one of those steps has been compressed. Not by adding people. By giving the people you have tools that take the grunt work off their plate so they can focus on the decisions that actually matter.

Engineers use AI-assisted coding daily — not to replace thinking, but to handle the scaffolding so they can spend more time on architecture and edge cases. Product specs that used to take a week to draft now take a day. User research synthesis that buried someone for a whole sprint now surfaces patterns in an afternoon. None of that is magic. It’s just less time on the mechanical parts of the job.

At a large company, these gains often get absorbed by process. The time you save drafting a spec gets eaten by the review cycle. The faster prototype still has to go through three governance checkpoints. The efficiency gains are real but they hit a wall of organisational overhead.

At a small company, there’s no wall. The time you save goes straight back into building.

If you’re at a small company right now and you’re not feeling this, I’d gently suggest the issue isn’t the tools. It’s how you’re using them (or perhaps not using them?). The teams I see getting the most out of Gen AI aren’t the ones asking “how do we add AI to our product.” They’re the ones asking “what took us two months that could now take us two days.”

There has never been a better time to be small and opinionated about what you’re building!

Last week the GSMA — the global body representing mobile network operators — launched an initiative to build AI models specifically for telecommunications. Not fine-tuned wrappers around existing models. Purpose-built, from the ground up.

Why? Yup, it’s the network data.

Frontier models applied to network data didn’t go so well. They invented non-existent frequency bands. They confused critical technical concepts like RAN slicing and network slicing. They sounded confident doing it.

An analyst from IDC put it well: a wrong answer in a customer service chatbot might affect one billing line item. A wrong answer in network operations could trigger a failure that cascades across thousands of customers — consumers and enterprises alike.

But here’s the question I think the GSMA might have overlooked. Is this really a model quality problem? Or is it a use case fit problem?

I think it’s the latter. And I think it’s a question more product teams need to sit with before they reach for an LLM.

We’ve got into a habit of treating large language models as the default starting point for any AI feature. Customer support? LLM. Internal search? LLM. Document processing? LLM. And often that’s the right call — these models are genuinely remarkable at language tasks.

But not every problem is a language problem. Telco network data isn’t unstructured text. It’s structured, numerical, vendor-specific, and governed by standards that run to thousands of pages. The reason frontier models struggled isn’t that they’re not smart enough. It’s that they were built to predict the next token in a sequence, and that’s just not what this task requires.

When you’ve got a hammer that writes poetry, everything looks like it needs a sonnet. The pressure to use generative AI — from boards, from investors, from the market — makes it genuinely difficult to stand up in a room and say “actually, this isn’t the right tool for this particular problem.”

But that’s the job. Choosing the right approach for the problem is a core product skill. As generative AI eats up the time-consuming work of writing PRDs (hurray!), judgment on picking the right approach rises in importance. And sometimes the right answer is: not this. Not yet. Not here.

Less than 25% of recognized diseases have approved drug treatments. Yet these treatmentless diseases impact hundreds of millions of people a year. And for some, treatments might actually already exist.

To quote from the Every Cure website “For instance, an inflammatory disease and a certain type of cancer might share common mechanisms in the body (e.g., related proteins and genes responsible for the disease) and thus may be able to be treated with the same drug.”

However, if a drug is off-patent and the disease is rare, there’s no financial incentive for a pharmaceutical company to run the trials. The drug already exists. The patients already exist. But the match between them never gets made, because there’s no business model that pays for the search.

This is a market failure, not a knowledge failure.

Every Cure is a nonprofit trying to close that gap. Founded by Dr David Fajgenbaum, a physician who was diagnosed with Castleman disease while in medical school and nearly died multiple times before identifying that sirolimus — a decades-old transplant drug — could treat his condition.

His team has since built MATRIX, an AI platform that systematically analyses all known drugs against all known diseases and ranks the most promising matches. In 2025, they reviewed over 9,000 repurposing opportunities and launched nine active programs targeting diseases where patients still lack effective treatment.

Phenomenal. But I think this story has an even deeper lesson to learn from.

We spend a lot of time in tech debating whether AI will replace jobs, disrupt industries, or become sentient. Most of that discourse is speculative. Every Cure is an example of something much more concrete: AI being used to solve a coordination problem that humans created but can’t solve at scale. The data exists. The drugs exist. The patients exist. What was missing was the capacity to search a 75-million-option space and prioritise the combinations worth testing.

That’s not artificial general intelligence. It’s not even particularly exotic machine learning. It’s pattern matching and prioritisation applied to a problem that matters enormously and that the market has no reason to solve on its own.

Ninety-five percent of the 7,000 known rare diseases have no FDA-approved treatment. Not because treatments are impossible, but because no one’s looked hard enough.

Sometimes the most important thing AI can do isn’t the most impressive thing. It’s just the most overdue.

I know a lot of people are AI curious but don’t know quite how to jump in. I’m no expert, but I’m happy to experiment and share what works and what doesn’t!

So this is the first in a series of three posts about how we’re using AI tools — specifically Claude Code and Claude Cowork — in the product development process at my new company, Tapi. Not as a novelty. As part of how we work.

Here’s what that session looked like.

I started by feeding Claude (CoWork) a doc our sales team had compiled during conversations with customers, some geo specific market research and some Slack discussion threads between the CX and Sales teams.

Then I connected it to our codebase. That last part is what changes things. The AI wasn’t working from abstractions — it was reading our actual schemas, our API routes, our business logic. When it proposed a data model, it referenced our existing patterns. When it said something didn’t exist yet, it had checked.

The first version of the PRD was solid but narrowly scoped. Then the team’s field notes surfaced a problem I hadn’t thought through deeply enough, and the second version took a fundamentally different approach.

Then I challenged the framing — the module was scoped to one market, but the capability underneath was market-agnostic — and the third version restructured around a cleaner architecture. Then a third codebase entered the picture and the fourth generation PRD had to account for a system that was more complex than originally described.

That’s the part I find most interesting. The AI didn’t produce those four versions by iterating on language. Each revision was driven by a conversation — me bringing a question or a challenge, the AI responding with analysis grounded in the code, and the document evolving as a result.

I want to be clear: what I didn’t say is “write me a PRD.” I’m fairly sure that would have produced garbage.

It’s a working session where I’m bringing product judgment and domain knowledge, and the AI is bringing the ability to hold more context than I can, move between codebases, and turn a conversation into a structured document in real time.

We’re building this into how we work at Tapi — not AI tooling as a separate step, but AI tooling embedded in the product thinking itself. More on what that actually looks like in practice over the next two posts.

Of course I went down that rabbit hole. Here’s what I found.

The problem RLMs solve is one most people working with AI have hit: the more information you feed a language model, the worse it gets. Even models with large context windows suffer from context rot — facts get jumbled, details in the middle of a long document get lost.

To date the best fix has often been a form of RAG — store your documents, retrieve the relevant chunks, feed those to the model. It works, but it’s a lot more fiddly than most people realise. You’re constantly tuning what gets retrieved and whether the right context made it into the prompt.

RLMs take a different approach. Instead of pre-processing documents and hoping you retrieved the right bits, the model gets access to the full input as an external environment it can interact with programmatically. It writes code to search, filter, and chunk the text itself, then recursively calls a smaller model to process each piece.

The paper comes out of MIT CSAIL, published in late 2025. The open source library is already available.

What I find most interesting is the design philosophy. Rather than making the model’s memory bigger — the brute force approach the major labs have been pursuing — RLMs make the model smarter about how it uses its memory. It’s the difference between giving someone a bigger desk and teaching them how to file.

Has anyone else experimented with this yet? Curious whether the results consistently hold up outside the benchmark.

Smith spent decades trying to answer a question that much of the tech industry still hasn’t properly asked: what is it that AI actually does? And how is it similar to and different from what humans do?

His answer, laid out most fully in his 2019 book The Promise of Artificial Intelligence, draws a line between two things we tend to blur together: reckoning and judgment.

To paraphrase him poorly, reckoning is what computers are extraordinary at. Pattern matching, calculation, classification, prediction. When an AI system reads a scan and flags an anomaly, that’s reckoning. When it processes a million maintenance requests and spots a trend, that’s reckoning. Fast, scalable, and genuinely impressive.

Judgment is something else entirely. It’s the capacity to weigh competing considerations in context, to understand consequences, to act with care when the stakes are real and the right answer isn’t in the training data.

Smith believed that judgment requires being in the world, not just modelling it.

We’re in an era where the word “reasoning” gets thrown around a lot. New model drops, new benchmarks, new claims about AI that can “think.” Smith would likely have pushed back hard on that framing. What these systems do — even the very best of them — is sophisticated reckoning. They’re getting better at it at a remarkable pace.

But reckoning and judgment are not points on a spectrum. From Smith’s perspective, they’re different in kind.

Smith’s framing has quietly become one of the most useful lenses I reach for when I’m working on AI products. Not because it tells you what to build, but because it tells you where to stop assuming the machine has got it covered.

But recently I realised that a number of the Substacks I enjoy reading have actually evolved towards a shorter form. So I’m trying something new.

If you follow me on LinkedIn as well as Substack, you’ll see duplicate content. Read it in the platform you prefer - and at least in Substack you won’t miss a post as the world goes rushing by.

Agriculture doesn’t get enough attention in AI conversations

But Australia is having a crack at changing that in Melbourne this week with evokeAg 2026 kicking off tomorrow.

Sadly I have not managed to find a solid excuse to go or the offer of a free ticket. But I’ve been vicariously attending by reading through the lineup and stalking the speakers.

The program promises to dive into “the uncomfortable, the uncertain and the urgently important”, including whether Australia can lead rather than be left behind in the AI-driven industrial revolution, whether agritech can shift the dial when the rain refuses to fall, and how Australian agriculture weathers an era of volatile, geopolitically charged markets. Big questions!

And for attendees, the opportunity to meet face to face and talk to the people who have gone past asking questions and into building solutions. Here are three that caught my eye.

Fiona Turner, CEO of Bitwise Agronomy, is speaking about GreenView — an AI-powered yield forecasting tool for berry and grape growers. Growers attach a GoPro to machinery they’re already using, capture video while they work, and get back a yield forecast within hours. It’s been trained on a huge number of images across 60+ berry varieties, and takes forecasting accuracy from around 50% with traditional methods to about 90%. Fiona built it after buying a vineyard in Tasmania and hitting the same data problems every grower hits — then applying 15 years of deep tech experience to solve them.

GrazeMate is showcasing an autonomous cattle mustering system. Drones trained on expert stockmen behaviour use reinforcement learning to move cattle calmly and independently. The grazier initiates a muster from a mobile app and gets a notification when it’s done. Founder Sam Rogers is a 19-year-old mechatronics engineer from North Queensland who just raised US$1.2 million through Y Combinator. If Bitwise is what happens when a tech veteran goes back to the land, GrazeMate is what happens when someone grows up on it and builds what they wish existed.

And Cropify is using computer vision to grade grain and pulses objectively, replacing the subjective human assessment that’s been the industry standard. A specific problem, a measurable improvement, and a clear reason why a machine does this particular job better than a person.

There are over 50(!) agrifood startups in the Startup Alley and a demo stage running across both days. Serious FOMO. If you’re attending, I’d love to hear what catches your eye.

The terminology shift that sticks in my head at the moment is a new second meaning I recently noticed for the phrase ‘human in the loop’.

Just put a human in the loop

‘Human in the loop’ has been around as a concept in machine learning / AI product build circles for a long time. It would typically come up in design discussions and risk mitigation workshops about the same time that folks were discussing that algorithmic decisions would only be correct ‘about 88% of the time’.

Someone would suggest that it was OK, we just had to keep a human in the loop to pick up the errors. On a reasonably savvy team, someone else would point out that humans are really not great at staying alert to errors that occur infrequently, and hopefully some combination of thoughtful UX design, graceful rollback from failure and acceptance of business risk from incorrect decisions would be found to sufficiently alleviate the problem.

But the cases where this didn’t happen and the responsibility of catching the AI mistakes got hung entirely on the poor ole ‘human in the loop’ have worried me in a compounding way over the years and still niggle today. (If you want to dive deeper, this 2019 paper by Madeleine Elish is my go to for illustrating the wider issues.)

Recently, teams working to create AI assistants that can undertake tasks on behalf of a user e.g. booking travel, buying groceries, have started using the same phrase ‘human in the loop’ in what seems to me a subtly different way. Rather than book my entire European holiday sight unseen, an AI assistant would do all the option hunting, price and schedule comparisons and then present me with the best complete itinerary for approval and probably payment authorisation.

What’s the difference that bugs me? I’m still trying to pin it down but I think it’s the frequency and the distinctness of the task. In traditional usage, the ‘human in the loop’ is in a high frequency business process loop like credit approval or defect management. This human in the loop therefore sees hundreds or even thousands of AI decisions in a working day and all the decisions are fairly similar and not personally relevant to the human. In the emerging usage, the human is assessing the draft result of a task that matters more to them as an individual and the task output is distinctive and engaging.

So what’s the problem? Fundamentally, that I think this new usage will shift slowly over time and morph into something much close to the old usage.

If AI Assistants get actually useful, they will become more ubiquitous and the ‘human in the loop’ decisions will reduce in distinctness. We then head back into the problem of trying to design for attention in a boring situation.

But the phrase ‘human in the loop’ will be baked into a new set of brains in the frame of an ‘interest engaging’ decision. I have a nagging feeling of trouble ahead.

Now what?

To offer something actionably useful to offset an angsty post, my advice on how to guard yourself against this language morphing blindness is to really mindfully not use jargon in conversation or written communication until you’re sure that you know what you mean AND that the other people in the conversation mean the same thing.

Jargon emerges as useful shorthand for a community of people who are talking about the same thing often. In that context, it’s really helpful. But when a group of people pick up jargon that inadvertently bridges a concept understanding / agreement gap, it gets really not useful, really fast.

As a concrete example, this is the reason I don’t currently use the term Agentic AI in casual conversation - there really is no usefully settled definition of this phrase that stretch beyond small groups of people working in particular labs or companies.

Yes conversations about what you are building / buying and how / for how much take longer if you constantly use 10 or 15 words instead of 2 (or is that 3?) but the cost of premature brevity can be high if you build yourself into a risky situation or buy a company based on a mutual misunderstanding.

Given that there is nothing particularly special about Australia (I mean, many special things of course which we love and you can share if you want to visit or move here, but we’re not anyone’s idea of a global superpower …) I assume that many, many similar conversations are happening around the world. The French have Mistral and Canada has Cohere.

I was drawn back to thinking about these conversations again because of the latest insanity from the Trump administration - a requirement that all Generative AI tools included in solutions seeking US Government contracts must prove (?!) they are ‘free of ideological bias’ aka anti-woke AI. (Translation: AI that only generates text/images that reflect Trump’s personal world view. Eek!)

Sure does make the idea of a viable alternative to a US domiciled Generative AI service increasingly attractive.

So in this post, I’m thinking out loud about the challenges I see / the reservations I have with Australia specifically jumping into this. Some points will be relevant to many countries, others likely more AU specific. My view on this is evolving, so keen to hear your thoughts in the comments.

Which bit needs to be sovereign?

To make a useful LLM from scratch, you need at a minimum

• a lot of data,

• a lot of computational power,

• some highly skilled software engineers and

• a lot of humans to perform the human bit of RLHF (reinforcement learning with human feedback)

Then if you want to run that model in a sovereign manner, you’ll continue to need a lot of compute. More on that later.

Exclusively Aussie data?

Let’s assume for a moment that you’re a purist, and you genuinely want to start from absolute scratch. I can’t see a way that the data you use can be sovereign, if by that we mean (and I know this is a bit hand wavey) sourced only from Australia. Without getting granular about what ‘Australian’ data would even mean, there just isn’t enough of it. These models only work with all the data you can hoover up. So any Australian company wanting to build a useful LLM from scratch would need to embrace global data.

I know that for some of the folks pursuing Australian LLMs, better fluency in indigenous languages, Australian idiom and Australian legislation is a key driver. A great motivation but probably most pragmatically achieved using fine tuning so it doesn’t require training an LLM from scratch.

So in my considered opinion, you can’t build a capable LLM using purely sovereign Australian data. We would hence have to pick a base model, likely one of the very functional fully open weights models. That makes it super likely you will either pick a model trained by a US organisation in a US context or a model trained by a Chinese organisation in a Chinese context. I don’t think either of those choices are particularly problematic (we’re picking up the weights, hosting this on shore and doing a whole bunch of additional tweaking right?) but it’s good to be clear that from the very beginning, this isn’t going to be a purists ‘sovereign model’.

Inherited choices

If we’re picking up a base model that has been trained elsewhere by other folks, let’s be clear from the outset that we are also picking up all the moral / ethical choices they’ve made along the way.

Whether some of the data used was taken without informed consent from the original creators, whether the energy used to run the training cycles was sourced from renewables or the brownest of coal, the working conditions of any data cleaners / content moderators / human trainers. You get my point.

Pragmatically, this is a decision every consumer of Gen AI services (organisations and individuals) makes everyday. Brutally, the most that most of us tend to do is feel a twinge of conscience, sigh, rail briefly against the unfairness of the world and use the service anyway.

But I think it’s good to be straight with ourselves that, if we’re picking up a base model, we’re not righting any wrongs or taking a higher moral ground.

Onshore computational power?

This for me was one of the earliest arguments that made me look twice at why some form of ‘sovereign AI’ capability might make sense for Australia. Any way you look at it, Generative AI models and applications and workflows built on top of them are computational hungry.

While traditionally AI has been expensive to train / cheap to run, Gen AI is gobsmackingly computationally expensive to train and eyewateringly computationally expensive to run. DeepSeek and Moonshot AI have proven that you can be a lot more frugal with GPU power if you really need to be but still, not a computationally light technology.

If you start to run any significant part of your economy on LLM based services (and I don’t think we need to approach anything like ‘AGI’ to get to that point) then, yes, you would want control over the continued, robust supply of that compute. This is a particularly salient one for Australia where a lot of our internet traffic shuffles back and forth across a very finite number of deep sea cables that sit on the ocean floor and might accidentally or on purpose be severed by a passing warship. If your payments provision or your healthcare notetakers or your freight routing, etc starts to rely even in part on LLM calls that go over the wire to an offshore data centre for fulfilment, what does ‘graceful failure’ look like in the event of an accidental or malicious loss of a cable? Might start to look pretty ungraceful.

Eight months ago, worrying about losing reliable internet service to the rest of the world would have sounded a bit far fetched and unlikely outside of communities where folks are professionally paid to worry about such things. Now I’m not so sure - seems worth thinking about quite a bit unfortunately.

And yes, I know there would be knock on to things other than LLM based services if the internet gets compromised. But we’re used to worrying about those things and so, hopefully, have backup plans of some kind? Heavy adoption of LLM based applications is a new risk and new things are notoriously tricky.

Highly (and very specifically) skilled software engineers?

Not saying we don’t have highly skilled software engineers in Australia, we do. But this is a very specific skillset that we’re talking about and while the tricks of the trade can definitely be learned, that takes time. Bringing currently ‘100% up to speed’ engineers ‘back home’ from the current talent hotspots? Mark Zuckerberg has recently given in a good case study in how much that is likely to cost.

Growing our own (takes time) and keeping them in Australia to work on sovereign AI? One for the ‘needs a pretty robust plan’ basket.

Ownership?

Who gets to invest in this Australian LLM? i.e. where does the considerable cash needs come from before there’s a marketable product? Only the government? Only ASX listed companies? Only domestically domiciled investors? I don’t even have the right vocabulary for this but you get my point. ‘Foreign’ ownership might open the door to non-sovereign influence. Is that acceptable?

Business model?

Let’s set aside for a moment that no one has yet come up with a convincing business model for LLMs where sustainable revenues stack up against incurred and ongoing costs. Maybe there is a ‘Google-esque’ advertising rabbit to be pulled out of hat somewhere. Maybe not.

Who wants to pay for an Australian sovereign AI model? Given that we still don’t really know what that term means, let me for the sake of argument take a punt and suggest we split it roughly into three flavours:

• global base model + AU specific data and tweaking

• hosted here on data centres that are in some meaningful way owned by Australia

• subject only to Australian regulation

Will those three points be meaningful enough to a large number of consumers that they will pay a price premium?

And the rest

There are lots of other interesting and some important questions into how sovereign is sovereign enough to achieve whatever it is you want to achieve.

• Is it OK to use GPUs that come from an overseas fab and an overseas company? Kinda gotta be as we’re not building chips here. So what risks are we accepting with that ‘choice’?

• Is it OK to use an operating system that isn’t Australian? This is like chips. No viable alternatives so note down some more risks.

• Same question about the many, many pieces of the software stack needed to train / fine tune / tweak / serve / secure the LLM models. We share a lot of things in this global economy.

So what?

Hey I did tell you right at the get go that this was just me thinking out loud.

To be clear, I haven’t started, joined or invested in a company building sovereign AI for Australia (yet). But this is the first tranche of questions I’d want to answer before I did.

So I’m trying something new today, a co-written post with . Mark featured in my AI reflections and futures series earlier this year and his Q&A was one of my most read January articles. So the rest of this post is all Mark. Hope you find it helpful.

So you’re getting into agents and think you might need an agent framework. But which to choose and is it better at this stage just to write what you need yourself, retaining complete understanding and control at the sacrifice of up front development time?

In a recent consultation with a client, my recommendation was DIY - keep it lean and modular because the then current frameworks were really light on and the space evolving rapidly. After some discussion, they went down the framework route. They had limited development resource to play with and, betting that agent frameworks were a robustly solved problem, chose to focus instead on the UI. Sadly they weren't able to achieve the consistent intelligence they were after and had to backtrack and start over.

Fast forward to today and not as much has changed as you might hope. Most agent frameworks define a workflow (RPA style) but those workflows are rigid and most are not able to handle non-trivial cases such as iteration, loop-back, etc.

For example, let's say you're creating a text2sql engine. The generated SQL is going to be wrong in some cases. A simple solution is to take the error message, feed it back in with the original input to try again. Many frameworks still struggle with this reentrant behaviour and this is far from a complex use case if you’re trying to build anything useful and non-trivial.

I would loosely define the levels of sophistication in agents as:

Level 1 Call a model with a custom system prompt ("Custom GPT")

Level 2 Draw a flowchart to create a sequence of steps and call tools where needed. (This is where MCP comes into play.)

Level 3 A more sophisticated workflow to handle more complex cases (loops, branching, etc.) - you need this pretty quickly for anything non-trivial

Level 4 Plan and Act Agents - similar to the current Deep Research Tools: use a model to create a plan given a goal/prompt, execute the plan steps (sequential or branch), synthesise the results and report back.

Level 5 Level 4 + human in the loop - request feedback from a human when required at stages in the flow

Level 6 Have multiple agents work on the problem: Coordinator, specialised workers/roles. (This is where A2A comes into play.)

Level 7 A larger network of agents, continuously running and monitoring.

Level 8 A "companion" anticipating needs and initiating any of the above as required.

That's about as far as practical at present. Most tools in the market, including stuff like Agentforce, are currently somewhere between Levels 2 and 4.

The coding agents (Cursor, Claude Code, etc) are more advanced and occupy Level 4 and 5 but for more narrowly defined problem spaces.

Level 6 could fairly be described as ‘under development’, and is not broadly useable for most applications or for teams starting out.

Levels 7 & 8 are largely aspirational but a path to implementation is visible.

Level 9 and above (AGI-ish and absent from my list but not at all absent from the influencer blogs) are hand-wavey speculation and probably years away from practical use. If that feels wrong to you, recall that as an industry, we've been talking about today's concept of AI agents for about three years and yet are still mostly at Level 2!

Tools like CrewAI, AutoGen and successors have been playing in Level 6. There are challenges of scale and enabling team-based development. Agent-to-agent (A2A) should play a role here. The current tools will likely be refactored to support A2A so will go through a bit of upheaval.

If I could, I would wait until things settle, and in the meantime, start lean with the other aspects of the platform required.

These tools can rapidly become fairly costly if you use them as a hosted service so don’t overlook that in your deliberations. Particularly when you couple that with the increase in LLM calls that go along with breaking down complex activities into smaller tasks. Cost and performance at scale become key considerations, particularly if you need to offer country specific processing.

Finally, the debate over when to use sequential agent calls vs agent networks isn't even close to settled. The topic of another blog post …

The three stages of AI progress according to a newish long read from and in their regularly excellent Substack newsletter, .

We use invention to refer to the development of new AI methods—such as large language models—that improve AI’s capabilities to carry out various tasks. Innovation refers to the development of products and applications using AI that consumers and businesses can use. Adoption refers to the decision by an individual (or team or firm) to use a technology, whereas diffusion refers to the broader social process through which the level of adoption increases. For sufficiently disruptive technologies, diffusion might require changes to the structure of firms and organizations, as well as to social norms and laws.

AI as normal technology, , April 2025

I’m guessing that only a handful of readers of Data Runs Deep are working in the ‘invention’ stage of Generative AI.

But many will be working in and around organisations where ‘innovation’ and ‘diffusion’ - both early adoption and adaptation - is definitely a choice and in some industries, perhaps a necessity.

Encourage (but don’t mandate) uptake and experimentation

I’m not at all a fan of the recent flurry of CEO memos exhorting their teams to reach for AI as a default for every task.

There is something almost desperate in the tone of a number of the notes which makes me wonder how many hard driving executive teams with little understanding of either the technology or the cadence of research breakthroughs have overcommitted to efficiency savings on the promise of autonomous agents and workforces full of AI employees.

Productivity gains to date have been modest, in the few cases where people are actually striving to measure them.

My bet is that this reflects that we’re in the very beginning of diffusion. We have no idea yet what the equivalent of the factory floor transformation that finally enabled full adoption of the electric engine will be for Gen AI.

So, many organisations will benefit over the medium term from encouraging and rewarding active experimentation with new Gen AI based tools. What’s needed is a little more patience, and the willingness to invest in some tools and a fair bit of decent training. As my Kiwi/Aussie readers will appreciate, it won’t happen overnight but it will happen.

We need to embrace tinkering across the workforce, not drive people into active opposition of all things AI by forcing it down their throats.

One of the pure pleasures of thinking hard about something is the number of spinoff thoughts that result and how much more you have to think about by the end of it!

Thinking and then writing about AI notetakers turns out to be one of those things. So an update.

Usage patterns that work

Since writing the initial post about AI notetakers, I’ve had two very interesting conversations with people who love using their notetakers. And the interesting thing is that how they use them sidesteps many of the challenges of permissions, inaccuracy and hallucinations that have been worrying me.

(1) Asking permission

I was visiting my GP (General Practitioner aka Primary Care Physician / Family Doctor) on a recent Wednesday. After we sat down and exchanged pleasantries she asked me if it was OK if she recorded our conversation with her AI notetaker.

Fantastic. She asked! If she can, we all can.

Given my professional interest in this space, I asked which software it was - PatientNotes.app - whether they stored their data onshore in Australia and what the deletion policies were.

My (awesome) GP paused, considered, said that she didn't know for sure as she had been using it for a while now, but it was an Australian company and she believed the storage was in Sydney. She said she would definitely check and get back to me, I said I was happy for PatientNotes to listen in and thought no more about it.

Fast forward to the following Friday (just two days !) and my GP sent me the following.

Dear Kendra

I just wanted to let you know that I checked and can confirm that the data from
PatientNotes is stored domestically is Sydney.

You will no doubt understand this better than I do, but they say that records are deleted 30 days after they are made, and I always delete mine at the end of the week anyway.

The webpage is https://www.patientnotes.app/privacy-and-compliance

If you would like to check further.

Now quite frankly in awe of this busy GP, I did of course check out the PatientNotes.app website and I'm impressed. They have a page on Security and a page on Privacy & Compliance.

Easy to find, easy to read. Kudos to Darren Ross, Sarah Moran and the team at PatientNotes.app

(2) Reviewing immediately

After posting on LinkedIn about my GP interaction, I fell into conversation with a New Zealand based lawyer who is a confident and frequent user of AI tools in his practice. Like my GP, he uses the AI notetaker tool during a client/patient consultation in place of the handwritten notes he habitually used to take.

Crucially, and this was true of my GP as well, he reviews the transcript and summary IMMEDIATELY after the consultation and makes edits. Then he saves the edited work as a record, effectively building on a note taking practice he already had in place for many years.

Contrast this with the anecdotal behaviour of many recent AI notetaker users - saving the transcript / summary UNREAD as a record of the conversation to refer to later, reading the transcript / summary of a meeting they DID NOT ATTEND to catch up with the content.

An telling example of the AI + human system working better for some workflows than for others.

And then Granola!

Just as I was pushing publish on the original post, I was in a conversation where people were discussing AI notetakers and their occasional experience of joining a call where there were no other humans, just AI notetakers. Then one of the women mentioned that ‘at least you could see the notetakers’. She had been told there were now AI notetakers that didn’t appear in the call at all. And voila, I discovered Granola.

Um, yeah, so as my new lawyer friend had commented - it might be safest to now assume that you are being recorded at all times.

Post image credit to Lucas Alexander on Unsplash

Occasionally those calls were recorded - maybe an all hands team call for a globally distributed team where you just couldn’t find a time zone friendly slot for all involved. We would carefully check that all involved understood that the call was being recorded, the software itself popped up reminders, often an audible tone and a spoken warning as well.

Fast forward to today, I regularly find myself in calls with not one but several AI notetakers. I can’t remember the last time someone asked if it was OK if their notetaker joined the call or if everyone on the call was fine with their voice and words being recorded, transcribed and summarised.

Transcripts are pretty good, summaries leave a lot of room for improvement

While I haven’t used a AI notetaker myself, I’ve read the summaries from a few (as an aside, I do wonder if this unprompted sharing of the summary is a canny tactic to reduce pushback during adoption) and I remain unimpressed.

As anyone who has tried to produce comprehensive, accurate, readable summary notes of a multi topic, multi speaker conversation will know, it is not easy to do, so I’m not casting shade on the AI notetaker efforts. But they don’t get a pass mark yet, let alone a high grade.

The transcription by and large is pretty good (except for ongoing challenges with industry jargon and three letter acronyms). Presumably this is helped a lot by the nature of online meetings where there is usually an individual mic for each participant giving clear speaker seperation.

Where does the data end up?

I’ve never been in a meeting where the terms and conditions of the given notetaking software is shared with the meeting participants. Sure, this would be extremely cumbersome and I get that it, if forced, it could rapidly become a box ticking exercise akin to the ‘reject cookies’ choices that pop up on many AU websites. But it irritates me that this means we slide unthinkingly and unasked towards the low friction path of data being hoovered up.

Imagine if one of the more popular tools is recording the audio so they can take a little extra time after the call to produce a high quality summary. They hang onto that data because, well, they can. Or frankly, because someone in the company forgets to auto delete it, which is much more common than you might think. They get hacked. Now all that voice data is on the dark web. Deepfake paradise.

Switched on by default

This personal bugbear of mine suddenly got a lot of press in Australia this week when the New South Wales education department realised that the video conferencing feature in the Microsoft Teams platform had been recording student biometrics (face and voice) since it was switched on by default in March.

The feature was switched off in April and the profiles were deleted within 24 hours of the department becoming aware that voice and facial enrolment was enabled.

The Guardian, 20/5/2025

In the Guardian article, a quote from Rys Farthing, the director of policy and research at the research organisation Reset Tech Australia, sums up the tone of my uneasiness about AI notetakers perfectly.

Was this data used to train their AI after it was collected? Are we sure it wasn’t disclosed or shared while it existed, and that all copies of it have been deleted? Data is like toothpaste, it’s hard to put it back in the tube once it’s been collected.

Minutes that really matter

Call it attention bias, but the notetaker references just kept on coming this week with the Australian Institute of Company Directors releasing new guidance on the use of AI notetakers and adjacent technologies in board meetings.

While AI may improve efficiency, it should never replace the critical role of human oversight. Boards, governance professionals and management must have appropriate controls in place to preserve the integrity and accuracy value of board minutes.

AICD statement on Effective board minutes and the use of AI, May 2025

As a non executive director, albeit of a sizeable not-for-profit not an ASX listed business, I am acutely aware of how pivotal the contents of board meeting minutes can become in the event of a crisis. So the AICD advice isn’t surprising. Where do you draw the line though?

I’ve had many busy execs tell me they think meeting summaries are great because they can catch up on what happened in meetings they miss. If you’re just talking about a general vibe check, that might work well for you. But I’m personally uncomfortable with the idea of relying on them for anything substantive at this point in time.

In short

As is typically the case these days, it isn’t the technology that is presenting the real concern here, at least not for me. It’s the human choices.

For the deliberate users - which I’m sure include many of my readers, please be mindful of the inaccuracies, both obvious and subtle, in the meeting summaries you are relying on. And please think carefully about which meetings you invite AI into, and how you proactively facilitate others to ‘eject the bot’. Be really sure you understand the impact of power dynamics here too - if you’re the highest paid person in the room, does ‘yes I’m OK with your AI notetaker’ really hold any weight?

For the other meeting attendees - the audio recording of your voice and potentially your image, are being captured and may be being stored. Your views will be represented, potentially inaccurately in the generated summary. That might be fine, or it might not. But be informed and if you’re not OK with it in a given context, be confident that you’re not the only one and that asking someone politely to remove their AI notetaker is perfectly reasonable.

For the app developers - I’m not a hater, I do see the great potential of these applications, particularly for folks who find meeting participation or attendance challenging. It would go a great way to restoring my confidence in your software if you would

(a) make it easy for users to choose which meetings to use a note taker in. When you opt it in by default to every meeting, make that hard or impossible to change, or offer the option to have it operate in stealth mode so that other participants don’t realise it is there, you make me very suspicious as to your actual motives. Yes, audio and image data is like liquid gold. No it is not OK to take it without informed consent and it is definitely not OK to use it in any other way.

(b) find a clear and succinct way to summarise how the data captured in the meeting is used, where it is stored and when it is deleted. Yes I know that’s tricky UX to do well but you build complex software so I have every faith in your ability on this challenge too.

(From memory, this was just after the first DeepSeek release that got a lot of public attention. As you will likely remember, the response was sadly much more ‘they cheated!’ than ‘cool work, let’s build on it’.)

And we wondered - how many of the most influential contributors to AI research in the last 20 years, were actually born in America? I’ll wait while you think about it, or you can just skip down to the list below.

Influential researchers in AI in the last twenty years

First we need some idea of who ‘influential’ is in the research world. I complied the list below by listing top-of-mind people I came up with as impactful and then asking Claude, DeepSeek and Chat GPT as well. A researcher made it into the list if at least three of those four sources listed them.

Birthplaces of influential researchers in AI

And now for the big reveal, where were they born?

Ah, gosh, right.

Where do they live and work today?

For many of these folk, where they live today will be a reasonable approximation of where they did much of their impactful research.

Well will you look at that!

What about the next twenty years then?

From the data above, it’s not exactly a stretch to say that standout talent in AI research can be born pretty much anywhere. And that an outsized share of it has been drawn into the US because of their absolutely top drawer, well funded and well supported universities.

With the current administration laying waste to all of that, it does make you wonder which country is going to step up to the plate and reap the benefits for the next half century or so.