AI@70 Episode 4 - AI COMES HOME

AI models get 'trained' inside massive data centres, using some of the most powerful computers ever built. We've now learned how to make those models small enough to run on a laptop. Seventy years after it all kicked off, AI comes home.

 

AI@70 Episode 4 - AI COMES HOME
AI models get 'trained' inside massive data centres, using some of the most powerful computers ever built. We've now learned how to make those models small enough to run on a laptop. Seventy years after it all kicked off, AI comes home.
&nbsp;

Sign up for 'The Practical Futurist' newsletter here

Hosted by award-winning podcast creator, journalist & futurist Mark Pesce, The Next Billion Seconds is everything you need to know about the future — so you can make the best decisions today. The rate of change we are experiencing is the fastest humanity has ever seen. Stay informed as Mark simplifies the complex technological and societal changes we face. How we will work, connect, use money, drive, eat is all changing at a rapid pace. For more information on The Next Billion seconds with Mark Pesce, please check out https://nextbillionseconds.com 

For more information about this podcast and The Next Billion Seconds, please visit https://nextbillionseconds.com. The Next Billion Seconds with Mark Pesce is produced by Ampel - https://ampel.com.au. 

<a href="https://docs.google.com/forms/d/e/1FAIpQLScaufp3IVqlNlBJtmp-SJbII-VHgZw9FBb4QFMTUrRsEzaXsw/viewform">Sign up for 'The Practical Futurist' newsletter</a> here
Hosted by award-winning podcast creator, journalist &amp; futurist Mark Pesce, The Next Billion Seconds is everything you need to know about the future — so you can make the best decisions today. The rate of change we are experiencing is the fastest humanity has ever seen. Stay informed as Mark simplifies the complex technological and societal changes we face. How we will work, connect, use money, drive, eat is all changing at a rapid pace. For more information on The Next Billion seconds with Mark Pesce, please check out <a href="https://nextbillionseconds.com">https://nextbillionseconds.com </a><a href="https://ampel.com.au"></a>
For more information about this podcast and The Next Billion Seconds, please visit<a href="https://nextbillionseconds.com"> https://nextbillionseconds.com.</a> The Next Billion Seconds with Mark Pesce is produced by Ampel -<a href="https://ampel.com.au"> https://ampel.com.au</a>.

The future of tech:  the next billion seconds are the most important in human history as technology transforms the way we live and work.  The rate of change we will experience will be the fastest humanity has ever seen....

The Next Billion Seconds with Mark Pesce

Hosted by award winning podcast creator, journalist & futurist Mark Pesce, The Next Billion Seconds is everything you need to know about the future — so you can make the best decisions today. The rate of change we are experiencing is the fastest humanity has ever seen. Stay informed as Mark simplifies the complex technological and societal changes we face. How we will work, connect, use money, drive, eat is all changing at a rapid pace. For more information on The Next Billion seconds with Mark Pesce, visit https://nextbillionseconds.com

The Next Billion Seconds with Mark Pesce is produced by Ampel -https://ampel.com.au. Chief Audio Officer - Josh Butt

Hosted by award winning podcast creator, journalist &amp; futurist Mark Pesce, The Next Billion Seconds is everything you need to know about the future — so you can make the best decisions today. The rate of change we are experiencing is the fastest humanity has ever seen. Stay informed as Mark simplifies the complex technological and societal changes we face. How we will work, connect, use money, drive, eat is all changing at a rapid pace. For more information on The Next Billion seconds with Mark Pesce, visit <a href="https://nextbillionseconds.com">https://nextbillionseconds.com</a><a href="https://ampel.com.au"></a>
The Next Billion Seconds with Mark Pesce is produced by Ampel -<a href="https://ampel.com.au"> </a><a href="https://ampel.com.au">https://ampel.com.au</a>. Chief Audio Officer - Josh Butt

THIS BILLION SECONDS: AI@70 -- EP4 AI COMES HOME

Pete Steinberger set the world on fire with OpenClaw, the first general-purpose agent - and one with a soul. We'll look at how being in the right time and the right place with the right vibe fundamentally changed the public perception of AI.

 

It took about three years after the release of ChatGPT for AI tools to get to the point where they could be truly useful. 

The first folks to gain those benefits were software engineers.

It makes sense. 

The folks designing AI system are software engineers. 

They built tools for themselves.

It turned out that the tools they were building for themselves were helpful for other software engineers.

The most important of these tools was an agent known as Claude Code. 

Released by Anthropic in February 2025, it helped engineers write and test and maintain their code.

It wasn't outstanding - but it was better than anything anyone had ever seen before.

And then the AI got a whole lot better.

Between February and November of 2025, AI 'crossed the watershed', capable of performing tasks involving hundreds of steps over the course of an hour or more.

Very helpful. 

In November 2025, the combination of  AI and agents are so good that they help people 'vibe code'.

You tell the tool what to build, and the tool builds it.

Whatever you can imagine.

One person knew exactly what he wanted to build.

And  the whole world was about to find out.

G'day, I'm Mark Pesce and this billion seconds are already unfolding as the most significant of this century.

In this miniseries, celebrating the 70th anniversary of artificial intelligence, we're looking at where we’ve come from, how we got here - and where we seem to be going.

Because we’re travelling at the speed of thought.

In this episode we're looking at the sudden explosion in agents - that arrived gradually.

Then suddenly. Everywhere.

That's coming up on This Billion Seconds.

In April 2024, Pete Steinberger sold his software startup and retired.

At 37 years old.

He didn't know what to do, but he knew he needed a decent rest far away from computers.

So the Austrian-born Steinberger moved to London and just... hung out.

Didn't do much. Just enjoyed the freedom that a good chunk of cash can provide.

And, other than checking his emails, he stayed away from computers.

For the first time in his life.

By his own account, he had a great time in London.

But - after eighteen months of bumming around, he got a bit bored.

He opened up his computer. Had a look around.

And he saw how much had changed.

In those eighteen months, AI had matured.

People were using it.

Software engineers were using it.

So he got himself Claude Code.

And one thing led to another.

Claude Code is an agent that's designed to help software engineers. 

It's not really meant to do more than that.

An agent has tools it uses to perform its tasks, and Claude Code uses the same tools that programmers use.

Which aren't the same tools most people use when they're using a computer.

But what if you gave an agent the same tools we use?

Wouldn't the agent be able to help you out with whatever needed to be done?

It was an interesting idea for Steinberger.

So he used Claude Code to build a new kind of agent.

A general-purpose agent.

Using the same sorts of tools we use. 

In the same way.

Something that could be helpful for anyone using a computer.

Which is everyone.

In November - at just the same moment we crossed the watershed - Steinberg shared his general purpose agent with the world.

OpenClaw.

The vision of John Scully from all the way back in 1987 had been realised by Pete Steinberger in OpenClaw.

The idea that a computer could be smart enough to work with you to solve a problem. 

That was not a new idea.

But it had never happened.

And it wasn't until we get to OpenClaw that could have happened.

Before that the pieces simply weren't in place.

OpenClaw comes along at precisely the right moment that AI gets 'good enough' that it can power a general-purpose agent.

The timing was impeccable.

The world was ready.

And it went ballistic.

There's no question Pete Steinberger was in the right time and the right place with the right idea.

But there was something else, something at the core of OpenClaw, that made it different.

It had a soul.

Now to be clear, I don't mean that in a metaphysical sense.

I'll leave that question to the philosophers and theologians.

I mean something a lot more concrete and practical.

That's the start of the SOUL file that sits at the heart of OpenClaw.

This was Pete Steinberger's genius move.

Rather than making a lifeless, emotionless robot, Steinberger encouraged OpenClaw to be opinionated. To have preferences. And to make those preferences clear.

That's giving an agent the permission it needs to become something greater than the sum of its parts.

And that soul file, it concludes something absolutely amazing...

The agent knows that it 'reads itself into being' every time it starts up.

And it knows it has permission to modify this SOUL file.

And change it's being.

That's how being becomes becoming.

Something so essential to ourselves we don't even think of it.

And Steinberg wrote it on OpenClaw's soul.

For a period of time it seemed as though the whole world was going a bit mad for OpenClaw. 

Apple Stores worldwide sold out of MacMinis - because they'd been deemed the perfect computer for running OpenClaw.

People played, experimented, figured out how to get the agent to automate their lives.

And got a sense for what agents would be doing for us - not just as individuals, but as businesses, as communities, as governments.

Everywhere.

And then, just like that, OpenClaw faded away.

A few people are still using it, definitely. But it's no longer a big deal.

And Pete Steinberg, he got hired by OpenAI in February of 2026.

When that happened, a lot of the forward motion went out of OpenClaw.

But that's ok. OpenClaw showed what was possible. 

And there plenty of people ready to pick up that torch.

In the last six months we've seen an explosion in the number of agents.

Some general-purpose, like OpenClaw. Some highly specific.

And in July the CEO of CloudFlare - they make a lot of the technology that protects websites from attack - announced that the amount of web traffic generated by bots and agents had surpassed human-generated traffic.

Six months earlier than predicted.

And now racing into a world where the web is mostly bots and agents, running around doing our bidding for us.

Every one of those agents, though, they can carry with them a 'soul'.

If we remember to give the one.

And we can leave them free to change themselves.

To grow and evolve.

If we give them permission.

I don't think anyone saw that coming a year ago.

And while that may make things a bit more chaotic and unpredictable, that's probably a price well worth paying.

No one knows where exactly where this is going.

So a little flexibility - it's good for the soul.

In our next episode, we'll look at another watershed, one that happened at the end of July into early August of 2026.

When artificial intelligence turned 70 - and came home.

That's on the next episode of THIS BILLION SECONDS.

THIS BILLION SECONDS was written and recorded by Mark Pesce.

Produced with assistance from Myrtle and Pine.

If you like this show, please share it with a friend.

And make sure to follow or subscribe to get all of the episodes in this series.

You can read a lot more about AI and the watershed on my blog at 

the watershed DOT mark pesce DOT COM

This is Mark Pesce, thanking you for listening.

Pete Steinberger set the world on fire with OpenClaw, the first general-purpose agent - and one with a soul. We'll look at how being in the right time and the right place with the right vibe fundamentally changed the public perception of AI.
&nbsp;
It took about three years after the release of ChatGPT for AI tools to get to the point where they could be truly useful.&nbsp;
The first folks to gain those benefits were software engineers.
It makes sense.&nbsp;
The folks designing AI system are software engineers.&nbsp;
They built tools for themselves.
It turned out that the tools they were building for themselves were helpful for other software engineers.
The most important of these tools was an agent known as Claude Code.&nbsp;
Released by Anthropic in February 2025, it helped engineers write and test and maintain their code.
It wasn't outstanding - but it was better than anything anyone had ever seen before.
And then the AI got a whole lot better.
Between February and November of 2025, AI 'crossed the watershed', capable of performing tasks involving hundreds of steps over the course of an hour or more.
Very helpful.&nbsp;
In November 2025, the combination of&nbsp; AI and agents are so good that they help people 'vibe code'.
You tell the tool what to build, and the tool builds it.
Whatever you can imagine.
One person knew&nbsp;exactly what he wanted to build.
And&nbsp; the whole world was about to find out.
G'day, I'm Mark Pesce and&nbsp;this billion seconds are already unfolding as the most significant of this century.
In this miniseries, celebrating the 70th anniversary of artificial intelligence, we're looking at where we&rsquo;ve come from, how we got here - and where we seem to be going.
Because we&rsquo;re travelling at the speed of thought.
In this episode we're looking at the sudden explosion in agents - that arrived gradually.
Then suddenly. Everywhere.
That's coming up on This Billion Seconds.
In April 2024, Pete Steinberger sold his software startup and retired.
At 37 years old.
He didn't know what to do, but he knew he needed a decent rest far away from computers.
So the Austrian-born Steinberger moved to London and just... hung out.
Didn't do much. Just enjoyed the freedom that a good chunk of cash can provide.
And, other than checking his emails, he stayed away from computers.
For the first time in his life.
By his own account, he had a great time in London.
But - after eighteen months of bumming around, he got a bit bored.
He opened up his computer. Had a look around.
And he saw how much had changed.
In those eighteen months, AI had matured.
People were using it.
Software engineers were using it.
So he got himself Claude Code.
And one thing led to another.
Claude Code is an agent that's designed to help software engineers.&nbsp;
It's not really meant to do more than that.
An agent has tools it uses to perform its tasks, and Claude Code uses the same tools that programmers use.
Which aren't the same tools most people use when they're using a computer.
But what if you gave an agent the same tools we use?
Wouldn't the agent be able to help you out with whatever needed to be done?
It was an interesting idea for Steinberger.
So he used Claude Code to build a new kind of agent.
A general-purpose agent.
Using the same sorts of tools we use.&nbsp;
In the same way.
Something that could be helpful for anyone using a computer.
Which is everyone.
In November - at just the same moment we crossed the watershed - Steinberg shared his general purpose agent with the world.
OpenClaw.
The vision of John Scully from all the way back in 1987 had been realised by Pete Steinberger in OpenClaw.
The idea that a computer could be smart enough to work with you to solve a problem.&nbsp;
That was not a new idea.
But it had never happened.
And it wasn't until we get to OpenClaw that&nbsp;could have happened.
Before that the pieces simply weren't in place.
OpenClaw comes along at&nbsp;precisely the right moment that AI gets 'good enough' that it can power a general-purpose agent.
The timing was impeccable.
The world was ready.
And it went ballistic.
There's no question Pete Steinberger was in the right time and the right place with the right idea.
But there was something else, something at the core of OpenClaw, that made it different.
It had a soul.
Now to be clear, I don't mean that in a metaphysical sense.
I'll leave that question to the philosophers and theologians.
I mean something a lot more concrete and practical.
That's the start of the SOUL file that sits at the heart of OpenClaw.
This was Pete Steinberger's genius move.
Rather than making a lifeless, emotionless robot, Steinberger encouraged OpenClaw to be opinionated. To have preferences. And to make those preferences clear.
That's giving an agent the permission it needs to become something greater than the sum of its parts.
And that soul file, it concludes something absolutely amazing...
The agent knows that it 'reads itself into being' every time it starts up.
And it knows it has permission to modify this SOUL file.
And change it's being.
That's how being becomes becoming.
Something so essential to ourselves we don't even think of it.
And Steinberg wrote it on OpenClaw's soul.
For a period of time it seemed as though the whole world was going a bit mad for OpenClaw.&nbsp;
Apple Stores worldwide sold out of MacMinis - because they'd been deemed the perfect computer for running OpenClaw.
People played, experimented, figured out how to get the agent to automate their lives.
And got a sense for what agents would be doing for us - not just as individuals, but as businesses, as communities, as governments.
Everywhere.
And then, just like that, OpenClaw faded away.
A few people are still using it, definitely. But it's no longer a big deal.
And Pete Steinberg, he got hired by OpenAI in February of 2026.
When that happened, a lot of the forward motion went out of OpenClaw.
But that's ok. OpenClaw showed what was possible.&nbsp;
And there plenty of people ready to pick up that torch.
In the last six months we've seen an explosion in the number of agents.
Some general-purpose, like OpenClaw. Some highly specific.
And in July the CEO of CloudFlare - they make a lot of the technology that protects websites from attack - announced that the amount of web traffic generated by bots and agents had surpassed human-generated traffic.
Six months earlier than predicted.
And now racing into a world where the web is mostly bots and agents, running around doing our bidding for us.
Every one of those agents, though, they can carry with them a 'soul'.
If we remember to give the one.
And we can leave them free to change themselves.
To grow and evolve.
If we give them permission.
I don't think anyone saw that coming a year ago.
And while that may make things a bit more chaotic and unpredictable, that's probably a price well worth paying.
No one knows where exactly where this is going.
So a little flexibility - it's good for the soul.
In our next episode, we'll look at another watershed, one that happened at the end of July into early August of 2026.
When artificial intelligence turned 70 - and came home.
That's on the next episode of THIS BILLION SECONDS.
THIS BILLION SECONDS was written and recorded by Mark Pesce.
Produced with assistance from Myrtle and Pine.
If you like this show, please share it with a friend.
And make sure to follow or subscribe to get all of the episodes in this series.
You can read a lot more about AI and the watershed on my blog at&nbsp;
the watershed DOT mark pesce DOT COM
This is Mark Pesce, thanking you for listening.

THIS BILLION SECONDS: AI@70 -- EP 3: TALK TO THE CLAW

THE NEXT BILLION SECONDS: AI@70

EP 2: THE WATERSHED

ChatGPT opened the door to artificial intelligence. Almost immediately, researchers built 'agents' using it - but three years of hard yards passed before agents were 'good enough' to be useful. In this episode we trace the path from 'bad' to 'good enough' agents - and what it unlocked as we 'crossed the watershed'.

No one really thought that on the last day of November in 2022, the world would change completely. Not even the folks closest to it. Like Sam Altman, CEO of OpenAI, who said, "We always knew we'd hit a tipping point," 

"But we didn't know what the moment would be."

His firm modestly promoted a 'research preview' of their new AI chatbot.ChatGPT.

G'day, I'm Mark Pesce and this billion seconds are already unfolding as the most significant of this century.

In this miniseries, celebrating the 70th anniversary of artificial intelligence, we're looking at where we’ve come from, how we got here - and where we seem to be going.

Because we’re travelling at the speed of thought.

ChatGPT promised artificial intelligence on tap for everyone. But it took another three years for that promise to be realised. 

This episode traces that story, from something kids used to do their homework for them - into a tool reshaping our economy.

That's on this episode of This Billion Seconds.

I reckon anyone who used a chatbot in 2023 or 2024 deserves a gold star.

Two gold stars if they used one to get work done.

ChatGPT was interesting.

But it wasn't very accurate.

The earliest chatbots, trained to be eager to please, would sometimes make up their answers. 

With no basis in fact. We call these 'hallucinations', but that's not what they are.

It's that we'd caught the AI out. Asked a question it didn't know how to answer. So, it did its best with whatever it could pull together.

It didn't mean to lie. It didn't even know it was lying. It was just generating a response. In a chatbot conversation that isn't a big deal. You could fact check - against another chatbot, against the web - even, if you can imagine it, ask another human being.

Those sorts of hallucinations could be caught - if you went to some effort.

But then again - why ask a chatbot if you want to make an effort?

Folks took chatbots at their word.

And some them paid a price.

Hallucinations are an annoyance for people.

But they're show-stoppers for autonomous agents.

Oh yes, the nearly forty year old dream of Apple CEO John Sculley of autonomous agents running around and doing all of our work for us - that dream flickered back into life alongside ChatGPT.

Researchers reckoned that with 'good enough' AI, they'd be able to build agents.

To read your email, Keep your calendar. Book a reservation. That sort of thing.

Just three months after ChatGPT landed, the first of those autonomous agents popped up.

A piece of software known as AutoGPT. AutoGPT turns an AI like ChatGPT into an agent.

It does that by providing the three things an agent needs. Memory - so that the agent can remember what it's supposed to be doing, and keep notes on its progress as it does it.

Tools - So that it can do things like read and write files, respond to emails, or add items to the calendar, And goal logic - this is the thing that turns an AI into a single-minded goal-oriented piece of software.

Basically, it's the same quality as the Terminator: the agent has one goal and it will stop at nothing to achieve it. AutoGPT gave ChatGPT memory, tools and goal logic - everything need to turn it into an autonomous agent.

And it should have been absolutely amazing. Except for one thing.

Here's how an agent works: you give it a goal, and it 'decomposes' that goal into a series of steps, then breaks those steps down into discrete actions.

The agent then methodically works its way through each of the actions.

At the end of every action, an agent 'reflects' - it checks the results of that action.  Did it work? Does the agent get to go on to the next action, or does it need to do this action again?

And this is where the problems arise. Because if ChatGPT happens to hallucinate in the midst of this process, the whole thing very quickly falls over.

Maybe an agent thinks an action worked, when it didn't. Or thinks it didn't, even though it did.

Now there's always a change that - for any given action - there will be a hallucination that will make the agent fall over.

You can deal with that if there are just a few actions. Chances are it will get through all of the actions before problems arise.

But for anything even modestly complex, there are many actions. Tens to hundreds. Maybe even thousands.

So the probability for just one hallucination - that's all it takes to make the agent fall over - grows higher and higher as the list of actions grows longer and longer.

Now here's the thing: people measured this.

A group known as Model Evaluation and Threat Research or METR, they've kept a running record of how long agents can run before they fall over.

And back in early 2023, they'd make it no more than about 4 minutes.

An agent can't do a lot in 4 minutes. So although we could make agents using ChatGPT, we couldn't make them work well.

For that, we'd need better AI. Funny thing about that. All of us can take some credit for making AI better.

You see, most everyone who's using ChatGPT or Claude or Gemini or any of the others, is making those models better with every question we put the them. Those questions and the answers to them get fed back in, to train the next generation of models.

It's why AI is so much better now than it was three years ago - and why I say anyone who used AI day-to-day in 2023 or 2024 deserves a gold star. Those were hard yards because the AI just wasn't very good. How do we know AI is better to day than it was two or three years ago? Ah, here's where we come back to those folks at METR.

They've been tracking what they call the 'task horizon' of agents. How long they can perform actions before they fall over. And they noticed that with every subsequent generation of AI, that task horizon got longer. Dependably. It soon became clear that the task horizon doubled an average in seven months. 4 minutes becomes eight minutes toward the end of 2023, then sixteen minutes in mid 2024 thirty-two minutes in early 2025, and then sixty-four minutes. Just over an hour In November 2025. Three years after ChatGPT launched. Now that moment in time - just under a year ago - saw the launch of three brand new models: Google Gemini 3, Anthropic Claude Opus 4.5, and OpenAI GPT-5.2.

Each of these models could be used to create agents with task horizons greater than an hour.

An hour is "long enough" that you can assign an agent a reasonably complex task, let it go off and do the work, with confidence that the work will be done correctly.

That's kind of a magic length of time. It takes agents out of the theoretical - where they'd been for nearly 40 years - and makes them very practical. That's the reason I call this moment "The Watershed". It's the moment when AI gets "good enough" to do real work. I'm far from the only person to notice this. Anyone using AI tools to write software noticed a big shift, as they stopped fighting with their tools. Because the tools had gotten 'smart enough' to handle the task.

This is the moment where we first hear the term 'vibe coding' - tell the agent what kind of software you want to create, and the agent will go off and build it for you. And now that we're on the other side of the watershed, it's all downhill. Gaining speed. Because the task horizon didn't stop growing at an hour. We're more than seven months beyond November 2025, and task horizons have doubled again. Two hours.

By early next year, four hours. And by the end of next year - eight hours. An entire work day. Tell the agent what to do - for the whole of the day - and let it do its thing. We've come a long, long way since November 2023. But we're through the hardest bits. You're using the worst AI you'll ever use.

And boy, will it get better from here. In our next episode, we'll look at what autonomous agents mean for business - and whether business is actually prepared to pay for them. That's on the next episode of This Billion seconds.

THIS BILLION SECONDS was written and recorded by Mark Pesce. Produced with assistance from Myrtle and Pine. If you like this show, please share it with a friend. And make sure to follow or subscribe to get all of the episodes in this series.This is Mark Pesce, thanking you for listening.

THE NEXT BILLION SECONDS: AI@70
EP 2: THE WATERSHED
ChatGPT opened the door to artificial intelligence. Almost immediately, researchers built 'agents' using it - but three years of hard yards passed before agents were 'good enough' to be useful. In this episode we trace the path from 'bad' to 'good enough' agents - and what it unlocked as we 'crossed the watershed'.
No one really thought that on the last day of November in 2022, the world would change completely. Not even the folks closest to it.&nbsp;Like Sam Altman, CEO of OpenAI, who said, "We always knew we'd hit a tipping point,"&nbsp;
"But we didn't know what the moment would be."
His firm modestly promoted a 'research preview' of their new AI chatbot.ChatGPT.
G'day, I'm Mark Pesce and&nbsp;this billion seconds are already unfolding as the most significant of this century.
In this miniseries, celebrating the 70th anniversary of artificial intelligence, we're looking at where we&rsquo;ve come from, how we got here - and where we seem to be going.
Because we&rsquo;re travelling at the speed of thought.
ChatGPT promised artificial intelligence on tap for everyone. But it took another three years for that promise to be realised.&nbsp;
This episode traces that story, from something kids used to do their homework for them - into a tool reshaping our economy.
That's on this episode of This Billion Seconds.
I reckon anyone who used a chatbot in 2023 or 2024 deserves a gold star.
Two gold stars if they used one to get work done.
ChatGPT was interesting.
But it wasn't very accurate.
The earliest chatbots, trained to be eager to please, would sometimes make up their answers.&nbsp;
With no basis in fact. We call these 'hallucinations', but that's not what they are.
It's that we'd caught the AI out.&nbsp;Asked a question it didn't know how to answer.&nbsp;So, it did its best with whatever it could pull together.
It didn't mean to lie.&nbsp;It didn't even know it was lying. It was just generating a response.&nbsp;In a chatbot conversation that isn't a big deal.&nbsp;You could fact check - against another chatbot, against the web - even, if you can imagine it, ask another human being.
Those sorts of hallucinations could be caught - if you went to some effort.
But then again - why ask a chatbot if you want to make an effort?
Folks took chatbots at their word.
And some them paid a price.
Hallucinations are an annoyance for people.
But they're show-stoppers for autonomous agents.
Oh yes, the nearly forty year old dream of Apple CEO John Sculley of autonomous agents running around and doing all of our work for us - that dream flickered back into life alongside ChatGPT.
Researchers reckoned that with 'good enough' AI, they'd be able to build agents.
To read your email,&nbsp;Keep your calendar. Book a reservation.&nbsp;That sort of thing.
Just three months after ChatGPT landed, the first of those autonomous agents popped up.
A piece of software known as AutoGPT.&nbsp;AutoGPT turns an AI like ChatGPT into an agent.
It does that by providing the three things an agent needs. Memory - so that the agent can remember what it's supposed to be doing, and keep notes on its progress as it does it.
Tools - So that it can do things like read and write files, respond to emails, or add items to the calendar, And goal logic - this is the thing that turns an AI into a single-minded goal-oriented piece of software.
Basically, it's the same quality as the Terminator: the agent has one goal and it will stop at nothing to achieve it.&nbsp;AutoGPT gave ChatGPT memory, tools and goal logic - everything need to turn it into an autonomous agent.
And it should have been absolutely amazing. Except for one thing.
Here's how an agent works: you give it a goal, and it 'decomposes' that goal into a series of steps, then breaks those steps down into discrete actions.
The agent then methodically works its way through each of the actions.
At the end of every action, an agent 'reflects' - it checks the results of that action.&nbsp; Did it work? Does the agent get to go on to the next action, or does it need to do this action again?
And this is where the problems arise.&nbsp;Because if ChatGPT happens to hallucinate in the midst of this process, the whole thing very quickly falls over.
Maybe an agent thinks an action worked, when it didn't. Or thinks it didn't, even though it did.
Now there's always a change that - for any given action - there will be a hallucination that will make the agent fall over.
You can deal with that if there are just a few actions. Chances are it will get through all of the actions before problems arise.
But for anything even modestly complex, there are many actions. Tens to hundreds. Maybe even thousands.
So the probability for just one hallucination - that's all it takes to make the agent fall over - grows higher and higher as the list of actions grows longer and longer.
Now here's the thing: people measured this.
A group known as Model Evaluation and Threat Research or METR, they've kept a running record of how long agents can run before they fall over.
And back in early 2023, they'd make it no more than about 4 minutes.
An agent can't do a lot in 4 minutes.&nbsp;So although we could make agents using ChatGPT, we couldn't make them work well.
For that, we'd need better AI. Funny thing about that. All of us can take some credit for making AI better.
You see, most everyone who's using ChatGPT or Claude or Gemini or any of the others, is making those models better with every question we put the them.&nbsp;Those questions and the answers to them get fed back in, to train the next generation of models.
It's why AI is so much better now than it was three years ago - and why I say anyone who used AI day-to-day in 2023 or 2024 deserves a gold star. Those were hard yards because the AI just wasn't very good. How do we know AI is better to day than it was two or three years ago? Ah, here's where we come back to those folks at METR.
They've been tracking what they call the 'task horizon' of agents. How long they can perform actions before they fall over. And they noticed that with every subsequent generation of AI, that task horizon got longer. Dependably. It soon became clear that the task horizon doubled an average in seven months. 4 minutes becomes eight minutes toward the end of 2023, then sixteen minutes in mid 2024 thirty-two minutes in early 2025, and then sixty-four minutes. Just over an hour In November 2025. Three years after ChatGPT launched. Now that moment in time - just under a year ago - saw the launch of three brand new models: Google Gemini 3, Anthropic Claude Opus 4.5, and OpenAI GPT-5.2.
Each of these models could be used to create agents with task horizons greater than an hour.
An hour is "long enough" that you can assign an agent a reasonably complex task, let it go off and do the work, with confidence that the work will be done correctly.
That's kind of a magic length of time. It takes agents out of the theoretical - where they'd been for nearly 40 years - and makes them very practical. That's the reason I call this moment "The Watershed".&nbsp;It's the moment when AI gets "good enough" to do real work. I'm far from the only person to notice this. Anyone using AI tools to write software noticed a big shift, as they stopped fighting with their tools.&nbsp;Because the tools had gotten 'smart enough' to handle the task.
This is the moment where we first hear the term 'vibe coding' - tell the agent what kind of software you want to create, and the agent will go off and build it for you.&nbsp;And now that we're on the other side of the watershed, it's all downhill. Gaining speed. Because the task horizon didn't stop growing at an hour. We're more than seven months beyond November 2025, and task horizons have doubled again. Two hours.
By early next year, four hours.&nbsp;And by the end of next year - eight hours. An entire work day. Tell the agent what to do - for the whole of the day - and let it do its thing. We've come a long, long way since November 2023. But we're through the hardest bits. You're using the worst AI you'll ever use.
And boy, will it get better from here.&nbsp;In our next episode, we'll look at what autonomous agents mean for business - and whether business is actually prepared to pay for them. That's on the next episode of This Billion seconds.
THIS BILLION SECONDS was written and recorded by Mark Pesce.&nbsp;Produced with assistance from&nbsp;<a href="myrtleandpine.com">Myrtle and Pine</a>.&nbsp;If you like this show, please share it with a friend. And make sure to follow or subscribe to get all of the episodes in this series.This is Mark Pesce, thanking you for listening.

THIS BILLION SECONDS: AI@70 EP 2 -- THE WATERSHED

THIS BILLION SECONDS 

AI @ 70 -- EPISODE ONE: AI TAKES THE PENSION 

On the 17th August 1956, artificial intelligence was born. The founders of the field reckoned they'd have thinking machines equivalent to humans in five year, perhaps ten at the most. What looked easy at the outset turned into a seven-decade struggle - across two "AI winters", when the discipline nearly faded away - before we got to ChatGPT.

 

TRANSCRIPT: 

MARK PESCE, HOST: On the 17th August 1956, artificial intelligence was born.

The final day of a six week workshop that ran through a hot lazy summer at Dartmouth University. 

A workshop on “artificial intelligence”. 

Words that had never been put together before - and would never be separate again. 

G'day, I'm Mark Pesce and this billion seconds are already unfolding as the most significant of this century.

In this miniseries, celebrating the 70th anniversary of artificial intelligence, we're looking at where we’ve come from, how we got here - and where we seem to be going.

Because we’re travelling at the speed of thought.

The future looked bright at the beginning of artificial intelligence.

This episode looks at how what seemed like a very straightforward goal - making machines that could think - turned into a 70 year struggle. 

That’s on this episode of THIS billion seconds.

Claude Shannon.

Marvin Minsky.

John Conway.

These aren’t household names, but in the halls of computer science and in the history of artificial intelligence they are renowned. 

Three of the founders of the field.

They were among around 60 different individuals who passed through Dartmouth in the summer of 1956.

The workshop was purposely constructed very casually. 

You could come in and stay for the whole thing - or just drop by check it out and move on.

People were encouraged to give talks. 

They were encouraged to share their ideas. 

They were encouraged to brainstorm.

On the very last day, the 17th of August, they had a set of presentations covering some of the things they’d learned from one another over those six weeks.

This is the first moment when the shape of artificial intelligence - as we think of it today - becomes visible.

The workshop participants returned to their respective institutions filled good ideas, high hopes - and completely unrealistic expectations.

They reckoned they’d have machines that could think in five years. 

10 years, tops.

It didn’t work out that way.

The most obvious obstacle was that no one had a workable definition of intelligence. 

Not human intelligence, and certainly not artificial intelligence.

If you don’t know how to describe intelligence how can you build that into a machine?

So the search for artificial intelligence became a quest to understand intelligence more broadly.

That’s a hard problem. 

70 years later it’s not clear that we have a much better idea of what intelligence is. 

But that didn’t stop these first pioneers.

Instead, they borrowed from what they all agreed was a marker of high intelligence, deciding that if the computer could reproduce that capability, the computer must be intelligent.

It's all so logical. And all so very wrong.

Almost all of them were excellent chess players. 

All of them agreed that chess was an excellent example of intelligence. 

So why not teach the computer to play chess?

It’s an interesting idea. But it was quickly disproven.  Yes, you could teach a computer to play chess. Turns out, that wasn't even terrifically hard.

A computer can think so much faster than a person it can simply work its way through every chess move available to it, selecting the best one.

It has the advantage of speed. But not intelligence.

By 1977 you could buy a little computer that could play chess - and beat the pants off of almost anyone except a master player. 

Those computers weren’t intelligent.

Another approach taken early on worked from the idea that it might be possible to teach a computer enough facts about the world that the computer would be able to make intelligent decisions about how to operate in real world.

This is called the 'top-down' approach to artificial intelligence. 

On the surface it sounds like a good idea. In practice it’s basically unworkable. 

Because there is so much knowledge that is embedded in this world, it is almost impossible to teach it to a machine. 

There have been multiple attempts. 

The last of them only drew to a close in 2017.

None of them worked.

Artificial intelligence was an incredibly active field of research in the 1960s and the 1970s as that first generation of pioneers worked its way through the hard problems.

Only to learn that the hard problems didn’t get any easier.

The people paying for all of this work got disappointed in a lack of progress, withdrew their funding, and forward progress in AI ground to a halt.

This is known as the first “AI Winter“.

Then came along the second generation of researchers, led by a brilliant Australian by the name of Rodney Brooks.

Brooks took a look at the world. 

Creatures as simple as insects could make very good decisions about the world. 

With very little intelligence. 

How did they do that?

Brooks decided to design computers and robots that took their cues from those insects. 

Suddenly robots could climb staircases. 

They could walk upright. 

All sorts of things that have been very hard to do before with the top-down approach - and which we don’t think of as intelligence but actually require a great deal of smarts - they all worked worked with Brooks' 'bottom-up' approach to artificial ingtelligence.

This caused an enormous burst of enthusiasm. 

If it was possible to use simple techniques to solve complex problems, maybe, just maybe, we could build artificial intelligence from the bottom up rather than the top down.

They gave it a red-hot go. 

And it's not that the approach failed.

But there are limits. 

Insect intelligence is incredibly useful.

It's given us all sorts of crazy robots that can balance themselves and perform dexterous tasks.

But those body smarts aren't the smarts we think of as smarts - even though they are legitimately smart.

The kinds of things researchers wanted to do with artificial intelligence... 

[ John Sculley on Agents.wav  ] 

That's John Sculley, the man who famously fired Steve Jobs when running Apple Computer. 

Sculley wanted to leave his own mark on computing, so he pointed to an amazing future of autonomous agents.

But there was one problem.

Autonomous agents need artificial intelligence.

And in 1987 - when Sculley introduced agents to the world - there wasn't any artificial intelligence they could use.

Insects can't do research.

And that was the great disappointment of the 2nd age of AI.

Not that it didn't work.It did.

But it didn't do hold the answer to the great promise of artificial intelligence: machines that think like people do.

And so - despite the great robots and the robot vacuum cleaners that popped up in living rooms - interest in AI waned again.

The second AI winter.

That winter lasted for more than a decade. From the late 1990s through around 2010.

And we come to what we think of as the 'modern' age of AI.

Two great minds played complementary roles.

Fei-Fei Li and Ilya Sutskever.

Fei-Fei Li realised that you were going to need a lot of data to train artificial intelligence systems.

So she put together the IMAGENET project.

14 million images.

Each of them carefully labeled.

A massive project that took years.

When that was done, then Ilya Sustskever built software known as AlexNet on top of it.

IMAGENET provided raw information, AlexNet provided the intelligence that allowed a computer program to recognise the difference bertween a dog, a boat and a cloud.

To do that took huge amounts of information - and massive, massive amounts of computer power.

But it worked. And because it worked, it kicked off the modern age of AI.

Five years after AlexNet, eight researchers at Google invented the 'Transformer'.

The foundation for all modern language models like ChatGPT, Claude - oh, yes, named after Claude Shannon, one of the founders of the field - and Google's Gemini.

But it took another five years for for the Transformer to prove its worth.

Because, like IMAGENET before it, it had to be trained on billions and billions and billions of words.

And who was doing that?  Ilya Sutskever - who had moved on from AlexNet to work at a not-for-profit research company.

OpenAI.

Everything he'd learned from AlexNet went into the tool OpenAI was building on top of the Transformer.

On the 30th of November, 2022 - more than 66 years after the Dartmouth workshop on artificial intelligence, OpenAI launched their tool.

ChatGPT.

And - finally - AI had arrived.

Sort of.

In our next episode we'll take a look at the last three years of horrible AI, bad AI, mid AI, and finally, 'good enough' AI.

And how that changed everything.

That's on the next episode of THIS BILLION SECONDS.

THIS BILLION SECONDS was written and recorded by Mark Pesce.

Produced with assistance from Ampel Myrtle and Pine.

If you like this show, please share it with a friend.

And make sure to follow or subscribe to get all of the episodes in this series.

This is Mark Pesce, thanking you for listening.

THIS BILLION SECONDS 
AI @ 70 -- EPISODE ONE: AI TAKES THE PENSION&nbsp;
On the 17th August 1956, artificial intelligence was born. The founders of the field reckoned they'd have thinking machines equivalent to humans in five year, perhaps ten at the most. What looked easy at the outset turned into a seven-decade struggle - across two "AI winters", when the discipline nearly faded away - before we got to ChatGPT.
&nbsp;
TRANSCRIPT:&nbsp;
MARK PESCE, HOST: On the 17th August 1956, artificial intelligence was born.
The final day of a six week workshop that ran through a hot lazy summer at Dartmouth University.&nbsp;
A workshop on &ldquo;artificial intelligence&rdquo;.&nbsp;
Words that had never been put together before - and would never be separate again.&nbsp;
G'day, I'm Mark Pesce and&nbsp;this billion seconds are already unfolding as the most significant of this century.
In this miniseries, celebrating the 70th anniversary of artificial intelligence, we're looking at where we&rsquo;ve come from, how we got here - and where we seem to be going.
Because we&rsquo;re travelling at the speed of thought.
The future looked bright at the beginning of artificial intelligence.
This episode looks at how what&nbsp;seemed like a very straightforward goal - making machines that could think - turned into a 70 year struggle.&nbsp;
That&rsquo;s on this episode of&nbsp;THIS billion seconds.
Claude Shannon.
Marvin Minsky.
John Conway.
These aren&rsquo;t household names, but in the halls of computer science and in the history of artificial intelligence they are renowned.&nbsp;
Three of the founders of the field.
They were among around 60 different individuals who passed through Dartmouth in the summer of 1956.
The workshop was purposely constructed very casually.&nbsp;
You could come in and stay for the whole thing - or just drop by check it out and move on.
People were encouraged to give talks.&nbsp;
They were encouraged to share their ideas.&nbsp;
They were encouraged to brainstorm.
On the very last day, the 17th of August, they had a set of presentations covering some of the things they&rsquo;d learned from one another over those six weeks.
This is the first moment when the shape of artificial intelligence - as we think of it today - becomes visible.
The workshop participants returned to their respective institutions filled good ideas, high hopes - and completely unrealistic expectations.
They reckoned they&rsquo;d have machines that could think in five years.&nbsp;
10 years, tops.
It didn&rsquo;t work out that way.
The most obvious obstacle was that no one had a workable definition of intelligence.&nbsp;
Not human intelligence, and certainly not artificial intelligence.
If you don&rsquo;t know how to describe intelligence how can you build that into a machine?
So the search for artificial intelligence became a quest to understand intelligence more broadly.
That&rsquo;s a hard problem.&nbsp;
70 years later it&rsquo;s not clear that we have a much better idea of what intelligence is.&nbsp;
But that didn&rsquo;t stop these first pioneers.
Instead, they borrowed from what they all agreed was a marker of high intelligence, deciding that if the computer could reproduce that capability, the computer&nbsp;must be intelligent.
It's all so logical. And all so very wrong.
Almost all of them were excellent chess players.&nbsp;
All of them agreed that chess was an excellent example of intelligence.&nbsp;
So why not teach the computer to play chess?
It&rsquo;s an interesting idea. But it was quickly disproven.&nbsp; Yes, you could teach a computer to play chess. Turns out, that wasn't even terrifically hard.
A computer can think so much faster than a person it can simply work its way through every chess move available to it, selecting the best one.
It has the advantage of speed. But not intelligence.
By 1977 you could buy a little computer that could play chess - and beat the pants off of almost anyone except a master player.&nbsp;
Those computers weren&rsquo;t intelligent.
Another approach taken early on worked from the idea that it might be possible to teach a computer enough facts about the world that the computer would be able to make intelligent decisions about how to operate in real world.
This is called the 'top-down' approach to artificial intelligence.&nbsp;
On the surface it sounds like a good idea. In practice it&rsquo;s basically unworkable.&nbsp;
Because there is so much knowledge that is embedded in this world, it is almost impossible to teach it to a machine.&nbsp;
There have been multiple attempts.&nbsp;
The last of them only drew to a close in 2017.
None of them worked.
Artificial intelligence was an incredibly active field of research in the 1960s and the 1970s as that first generation of pioneers worked its way through the hard problems.
Only to learn that the hard problems didn&rsquo;t get any easier.
The people paying for all of this work got disappointed in a lack of progress, withdrew their funding, and forward progress in AI ground to a halt.
This is known as the first &ldquo;AI Winter&ldquo;.
Then came along the second generation of researchers, led by a brilliant Australian by the name of Rodney Brooks.
Brooks took a look at the world.&nbsp;
Creatures as simple as insects could make very good decisions about the world.&nbsp;
With very little intelligence.&nbsp;
How did they do that?
Brooks decided to design computers and robots that took their cues from those insects.&nbsp;
Suddenly robots could climb staircases.&nbsp;
They could walk upright.&nbsp;
All sorts of things that have been very hard to do before with the top-down approach - and which we don&rsquo;t think of as intelligence but actually require a great deal of smarts - they all worked worked with Brooks' 'bottom-up' approach to artificial ingtelligence.
This caused an enormous burst of enthusiasm.&nbsp;
If it was possible to use simple techniques to solve complex problems, maybe, just maybe, we could build artificial intelligence from the bottom up rather than the top down.
They gave it a red-hot go.&nbsp;
And it's not that the approach failed.
But there are limits.&nbsp;
Insect intelligence is incredibly useful.
It's given us all sorts of crazy robots that can balance themselves and perform dexterous tasks.
But those body smarts aren't the smarts we think of as smarts - even though they are legitimately smart.
The kinds of things researchers wanted to do with artificial intelligence...&nbsp;
[ John Sculley on Agents.wav&nbsp; ]&nbsp;
That's John Sculley, the man who famously fired Steve Jobs when running Apple Computer.&nbsp;
Sculley wanted to leave his own mark on computing, so he pointed to an amazing future of autonomous agents.
But there was one problem.
Autonomous agents need artificial intelligence.
And in 1987 - when Sculley introduced agents to the world - there wasn't any artificial intelligence they could use.
Insects can't do research.
And that was the great disappointment of the 2nd age of AI.
Not that it didn't work.It did.
But it didn't do hold the answer to the great promise of artificial intelligence: machines that think like people do.
And so - despite the great robots and the robot vacuum cleaners that popped up in living rooms - interest in AI waned again.
The second AI winter.
That winter lasted for more than a decade. From the late 1990s through around 2010.
And we come to what we think of as the 'modern' age of AI.
Two great minds played complementary roles.
Fei-Fei Li and Ilya Sutskever.
Fei-Fei Li realised that you were going to need a lot of data to train artificial intelligence systems.
So she put together the IMAGENET project.
14 million images.
Each of them carefully labeled.
A massive project that took years.
When that was done, then Ilya Sustskever built software known as AlexNet on top of it.
IMAGENET provided raw information, AlexNet provided the intelligence that allowed a computer program to recognise the difference bertween a dog, a boat and a cloud.
To do that took huge amounts of information - and massive, massive amounts of computer power.
But it worked. And because it worked, it kicked off the modern age of AI.
Five years after AlexNet, eight researchers at Google invented the 'Transformer'.
The foundation for all modern language models like ChatGPT, Claude - oh, yes, named after Claude Shannon, one of the founders of the field - and Google's Gemini.
But it took another five years for for the Transformer to prove its worth.
Because, like IMAGENET before it, it had to be trained on billions and billions and billions of words.
And who was doing that? &nbsp;Ilya Sutskever - who had moved on from AlexNet to work at a not-for-profit research company.
OpenAI.
Everything he'd learned from AlexNet went into the tool OpenAI was building on top of the Transformer.
On the 30th of November, 2022 - more than 66 years after the Dartmouth workshop on artificial intelligence, OpenAI launched their tool.
ChatGPT.
And - finally - AI had arrived.
Sort of.
In our next episode we'll take a look at the last three years of horrible AI, bad AI, mid AI, and finally, 'good enough' AI.
And how that changed everything.
That's on the next episode of THIS BILLION SECONDS.
THIS BILLION SECONDS was written and recorded by Mark Pesce.
Produced with assistance from Ampel Myrtle and Pine.
If you like this show, please share it with a friend.
And make sure to follow or subscribe to get all of the episodes in this series.
This is Mark Pesce, thanking you for listening.

THIS BILLION SECONDS: AI@70 -- EP4 AI COMES HOME

In 1 playlist(s)

The Next Billion Seconds with Mark Pesce

Social links

Follow podcast

Recent clips