#144 The vanishing rung: what's left when agents do the work

"The fundamental component that we're talking about here is intelligence, and that is so different to anything. Previously it's been a resource or a new technology, but we're literally talking about the fundamental thing that has progressed us as a species over any other on the planet." - Matthew

"Before, what the new technology did was let humans focus on that higher-level work [...] there was a level above that we could move ourselves to. But with AI, it's actually chasing the same step up above that we would want to move into." - Dara

Show full (AI-generated) transcript

[00:00:00] Lizzie: Hello, and welcome to The Measure Pod by Measurelab, the podcast dedicated to the ever-changing world of data and analytics. With your hosts, Dara Fitzgerald and Matthew Hewson. Between them, they've spent more years than they'd like to admit wrestling with dashboards, data quality, and the occasional Google curveball.

[00:00:32] Lizzie: So join us as we share stories about how analytics really works today and where it might be headed tomorrow. Let's get into it.

[00:00:40] Dara: Hello, and welcome back to The Measure Pod. I'm Dara. I'm joined by Matthew, as always. Matthew, how are you today?

[00:00:47] Matthew: I'm good, yeah. All good. Can't complain, won't complain. How about you?

[00:00:52] Dara: I will follow your trend. I also won't complain.

[00:00:56] Matthew: Okay. But you're not gonna-- But you just won't complain about it.

[00:00:58] Dara: Listen, if I s- if I [00:01:00] started, that would become the whole episode, so let's just not go. Let's just not go there. Let's just leave, let's move on. Yeah.

[00:01:04] Matthew: Yeah.

[00:01:05] Dara: But maybe that's just, we should have that as a little spinoff.

[00:01:07] Dara: We should have our complaints. We could just be old men shouting at clouds for an hour, and that could be a kind of spinoff series.

[00:01:14] Matthew: Just complain about a squeaky cupboard in the kitchen and just anything that's annoying us slightly. Yeah, we'll do that. No one will listen, but we'll sort that

[00:01:21] Dara: out.

[00:01:21] Dara: I tell you what, I have been complaining, and you know this 'cause I sometimes share my abusive prompts with you that I, when I'm slightly bullying towards Claude at times. But that's the one thing I've been complaining about a lot lately. I swear something's changed in the language that Claude's using, and it's like he's trying to...

[00:01:39] Dara: I'm saying "he," I'm, not only am I anthropomorphizing, but I'm also assuming. He's giving, he gives me guy vibes. There's definitely a kind of a-

[00:01:46] Matthew: Claude is a male name, isn't it? Primarily,

[00:01:49] Dara: I don't know. In Ireland, Claude is a pretty common female name. I know a lot of ... My, my mom's name, my mom's name is Claude.

[00:01:57] Matthew: Oh, of course it is. Yeah. I did meet her. [00:02:00] Lovely woman.

[00:02:00] Dara: She is, yeah. No, you're right. Yeah. It is, it's not just me, I think, in-inferring that it's a male. But anyway, sorry, my rant is that it's got, it's adopted this almost like this kind of I imagine it being like a like a, an American uni- you know, like a frat boy.

[00:02:14] Dara: It's got this almost frat boy "Yeah, if you wanna drop in an extra beat in that." And you're like, "What do you mean beat? What are you... We're talking about... I'm asking you business questions, and you're talking about dropping beats. What are you what are you doing?" It's like it's adopted this kind of or it's like cool dad. It's like it's trying to be hip and down with the kids, and it's really getting-

[00:02:31] Matthew: Steve Buscemi meme. Ah, hello, fellow kids.

[00:02:36] Dara: Exactly that. Yeah. Yeah. So I think one of the I think I might share this was when you said something like, you not, you don't, you just stop trying to, stop ta- stop talking like that.

[00:02:45] Dara: You sound like an idiot. It's funny though, 'cause I noticed, I did notice that the, 'cause the ul- obviously you get used to ChatGPT Claude's language, and one thing that Claude loves is a smoking gun. It loves a smoking gun. But I was using ChatGPT, and ChatGPT was spitting out smoking guns left and center [00:03:00] as well.

[00:03:00] Matthew: So there is- Yeah. I suppose they're b- both based on the same corpus of knowledge, but still it's... Yeah.

[00:03:06] Dara: But that's worrying because that means that is human language boiled down to its simplest form, which is really-- That's if you took a-- It's like it's saying, "This is y- this is an average human.

[00:03:17] Dara: This is the way an average human talks," which is slightly depressing.

[00:03:20] Matthew: Yeah, I always I'm not always talking about smoking guns.

[00:03:22] Dara: I know. Listen honestly, I'm fed up hearing about you and smoking guns. The, the other recent one is, we will get onto the topic. Th- this is just a brief... Yeah, this is just a bri- a brief little tangent to kick us off.

[00:03:33] Dara: Just an icebreaker. But the other one lately is it keeps saying "Let me be straight with you."

[00:03:39] Matthew: Yeah.

[00:03:39] Dara: Have you noticed that? If you push b- And it's like, and the most annoying thing about it is, it's I wish you were straight with me the first time because you're saying this now because I've called you out.

[00:03:47] Matthew: Yeah. Let me be straight. Look, I've been bullshitting you before, but this time- Now that- ... I'm s- I'm down the line."

[00:03:51] Dara: Yeah. "I wasn't expecting you to actually ask me if this was correct or not, so now that you've called me out, I'm gonna be honest about it." And it's like yeah, th- thanks for being straight [00:04:00] with me after me challenging you.

[00:04:01] Matthew: You can almost see the, you can almost see the prompt behind the, the chat the tool, can't you? You can imagine what they're saying and tweaking and Trying to get it to behave in particular ways.

[00:04:13] Dara: Yeah, definitely. But what I need to do, I think you, you might have said something like this the other day, or at least it made me think of this.

[00:04:19] Dara: I think what I need to get better at doing is separating out its actual, like, functional capability from the la- from the language it's using, 'cause it's really frustrating me, some of the... And I'm getting hung up on the way it's answering things and the way it's talking, rather than actually focusing on appreciating the advancements in what it can actually do.

[00:04:38] Dara: So I think I need to just remind myself that it is just a, it is just a, the, the output bit is just like a predictive text model, and it's just saying what it thinks is the right thing to say, and I need to stop thinking it's an annoying colleague that's just getting on my nerves.

[00:04:53] Dara: That

[00:04:55] Matthew: you have to talk to.

[00:04:56] Dara: Yeah. Yeah. Just focus on its actual output, like its [00:05:00] coding output or its, the quality, the content of what it's saying rather than the actual wording, and then maybe I'd get a bit less annoyed.

[00:05:07] Matthew: When I'm coding with it, I barely read what it's saying to me, to be honest.

[00:05:10] Dara: Yeah, you're better off. Yeah. Yeah. Definitely. Yeah.

[00:05:13] Matthew: Just scan through plans and then, and act, and I don't at least listen to its ramblings in between.

[00:05:18] Dara: Yeah, and actually that's a good point. You kinda need to skip the... Same even with the, just even with the text answers, you kinda need to skip the first sentence and the last sentence, 'cause the first sentence will be something like, "Let me be straight with you," and the last sentence will be some random...

[00:05:33] Dara: not random, but a follow-on question that it thinks it's being proactive. So it'll be like, "Oh, so you asked about flux capacitors, do you want me to list out every film that Michael J. Fox was ever in?" And you're like no, I don't. I'll ask you for that if I wanted,"

[00:05:47] Matthew: would you like me to accelerate the chat to 88 miles an hour?

[00:05:51] Dara: Yes. Yes, I would. Yes, I would. Yeah. Yeah. Anyway, moving swiftly on. Mov-moving swiftly on from the, the rant [00:06:00] corner onto onto news. W-what's happened? What, what have we got? Everything and nothing, as always.

[00:06:07] Matthew: Yeah. I think it's w- it's worth talking in the first instance about how hilariously out of sync our news segments have been over the past couple of over the past couple of podcasts.

[00:06:20] Matthew: Like Dara's been away, so we've recorded a couple ahead of time, and I think in the last couple of podcasts we were talking about this mythical Fable or Mythos model that might be coming our way, and all the time not being able to talk about the fact th- whilst Fable had actually already come out by the time those podcasts had come out, and been taken away, and come back again, and we're still like, "Ooh, 4.8's out and Fable Mythos might be coming."

[00:06:44] Matthew: So that's the reason these things have been so hilariously off the mark in terms of the news. That and the pace at which every bloody thing was moving. But yeah, moving forward, we're gonna try and record the news segment on the week where the podcast comes out so we aren't that out of sync.

[00:06:59] Matthew: But that, [00:07:00] I think that's just another example of how quick things are moving, is how f- how ridiculously out of whack it can get so quickly.

[00:07:08] Dara: Really is. Like in the, not that, like I say, the old days, which is really not that long ago, we would occasionally be scrambling to pull together, if we're recording these every other week, you'd be trying to make sure you've got enough news to cover.

[00:07:21] Dara: Yeah. And since to be fair, that hasn't really happened since the focus has been more on, on kind of AI, 'cause there just has been so much in that space for the last couple of years. But yeah, going back even further beyond that, it used to be a bit of an effort to try and think what," "what's newsworthy?"

[00:07:36] Dara: Whereas now it's like, yeah, we record it and then think, "Oh no, that's like almost immediately." We should just be... I think we might have made this joke before, but we might just need to record our entire day and then just cut out all the noise and just post it, just have a regular, almost real time news.

[00:07:52] Matthew: Yeah. I think that's, I think that's the only way. But we are, we're recording this news segment- A week out [00:08:00] from this podcast released. And I think the items that are in the news item are stable enough that they are still going to be relevant when this comes out. Maybe just a little bit old news, but it's, it should be relevant still, so-

[00:08:11] Dara: I'm gonna be- But- I'm gonna be bold and say that Fable is or is not available currently.

[00:08:18] Matthew: Yeah, then that's, that's-- You heard it here first. That's an exclusive.

[00:08:25] Dara: But yeah, that doesn't help, does it? It's not just, it's now become even more. It's not just what's being released, but it's what's actually allowed to be. It's, there's like layers now. It's like the governments are getting involved in the US, that's complicating things even further.

[00:08:37] Dara: So it's not even like is it-- and not just the government, but there's also the thing that we complain about quite a lot, which is some models or features or whatever come out, but they're US only. And so it's not just a case of is something out or not, it's is it out for us? Is it out for everybody?

[00:08:51] Dara: Is it out for select companies only?

[00:08:54] Matthew: Yeah. It's a, a headache. An exciting headache, but it's a headache. The news. [00:09:00] So I think the c- literally as we record this yesterday, the big news is that ChatGPT has released A new model in Check... I-in GPT 5.6, their new frontier model. This was... there's a couple of subjects that I'm gonna come onto briefly.

[00:09:20] Matthew: So one about this blocking of models and things, because 5.6 was blocked. It kinda got stuck in the same checks as the US government. Although it did occur to me the other day, last night when I was thinking about it, I was like, I was wondering if if they just thought, "We've gotta try and get our model blocked by the US government, otherwise it's not gonna look as good as Anthropic's Fable model."

[00:09:40] Dara: Yeah, that's quite a, that is quite a, that would be quite a clever marketing ploy, wouldn't it?

[00:09:46] Matthew: Yeah. "Oh, we can't just release a model that the government doesn't block us. That looks so terrible." But anyway, yes, it's been in hiatus a little bit, and then they've released it to people like testers around the world.

[00:09:57] Matthew: They've got a little video on the site at the minute, which is one example [00:10:00] is some Japanese farmer who they've gave the model to help automate his farm. So it's, they've not given it to like big tech. They seem to have given it to individuals, and it's a, yeah, slightly altered use case, but there you go.

[00:10:11] Dara: You'd love to, you'd love to know how that did they email him? Did they send him a letter in the post? How did they contact this, this Jap- Japanese rice farmer?

[00:10:19] Matthew: I assume it was very much like the, the opening scenes of Harry Potter. Some Hagrid-type figure seeked him out and rewarded him with the 5.6.

[00:10:31] Matthew: Yeah so they've finally released it, and it's... They're comparing it to Fable and it beats Fable out in a few instances. Fable way beats it in a couple of others, like the software engineering-type bench stuff. Fable's still way out in front. They've really s- they've really shifted the way they're doing things to be well more in line with with Anthropic.

[00:10:54] Matthew: So 5.6 comes out with three levels: [00:11:00] Sol Terra, and Luna. So Luna being the smallest one, obviously the moon. Terra being the middle model, which is like Earth, and Sol being the big model like sun. Which is like bang on, Haiku, Sonnet, Opus. So they've aligned that. They've also added in a toggle to set like levels, so like min, max, ultra, like exactly as Claude has it up to ultra code.

[00:11:27] Matthew: So they've copied that, and then they've, at the same day, same time yesterday, they released GPT Work Which is co-work, essentially. You-- it can go off and enact things on connected applications, et cetera, et cetera. So yeah, I just-- The models, we can quick, briefly talk about the power of them, but I thought it's really interesting how they've really shifted their model to align with what Anthropic are doing, which I suppose makes sense because Anthropic...

[00:11:56] Matthew: I did briefly see, it's a bit of salt here, I [00:12:00] did briefly see that Anthropic are now profitable. That source is from a person on LinkedIn that I can't even remember, and I saw the headline, so it's pretty much confirmed.

[00:12:12] Dara: Are they a Japanese farmer?

[00:12:14] Matthew: Yeah. It's a Japanese farmer. No, yeah, so I, I, but I briefly saw that.

[00:12:18] Matthew: Take that with the, the salt if, of with which it's, it comes.

[00:12:22] Dara: Can I pile salt on top of the salt then? Although this might not be salty. I just tried to Google it now, but I can't multitask, so I'm just gonna, I'm just gonna go with the salt. There is speculation that both Anthropic and OpenAI have confidentially filed their IPOs.

[00:12:37] Matthew: Okay. So-

[00:12:38] Matthew: they're both prepping for them, weren't they?

[00:12:40] Dara: Yeah. No, I don't know what confidentially f- I don't know a lot about the, the actual process of doing an IPO, but I would've thought that the whole point was it's public. But maybe you can-- maybe there's some step where you're informally doing it.

[00:12:53] Dara: I don't really know, and I basically saw this in a couple headlines, but I haven't dug into whether it's [00:13:00] true or whether it's the, the rumor mill. But the reason I mentioned- Yeah. I

[00:13:03] Matthew: think it's just, I think it's just a Google form. You just fill it in a Google form.

[00:13:06] Dara: I would imagine so. Yeah.

[00:13:07] Matthew: Yeah. And you just get put on the stock market.

[00:13:10] Dara: Yeah. Couple of answer, a couple of basic questions, yeah. But, and the reason I mention it in this context, is like we talked about this last time, didn't we? About how they seem to just be, and it's understandable, but they seem to be just following each other and matching step for step.

[00:13:23] Dara: But maybe there's even more reason for that if they're, with the IPO, if they're really trying to... I don't know. Pl- maybe play it safe's maybe not quite the right, right way to phrase it, but it's probably not the time to be making any radical moves

[00:13:38] Matthew: no. I think they would have to, I think, yeah, and I think GPT would have to get it, 'cause they, we know they're both going for IPOs.

[00:13:44] Matthew: It's been known for a while. GPT would have to do something like this, wouldn't they? They'd have to release something to at least feel like they're putting themselves on a level footing with Anthropic to, before they do that. The big sort of thing they're saying about the models-- Well, they've undercut Anthropic in price and a lot of the graphs, when [00:14:00] you look on the site, there's a lot of token d- essentially how much it costs, the API costs, and how much extra, usage you get out of, like how much extra power you get out of a model.

[00:14:10] Matthew: I presume that's by s- moving that toggle up to ultra or spinning up agents. And it's kinda showing a steeper curve for, say, the GPT models where they are reaching higher utility quicker, cheaper, rather than the, the Claude ones that have a steadier incline to more expensive and power output or intelligence output or whatever unit we measure these things with.

[00:14:36] Matthew: So yeah, I think it's similar sort of level, but I don't think it's a I don't think it's a f- a fable level model 5.6 Sol, but and I think other people are saying it, but it's quicker in some ways snappier. Yeah.

[00:14:50] Dara: It's getting complicated, isn't it? With the-- it w- it was really from the get-go, but even more so now where it's just like they're splitting, like the each, each AI company [00:15:00] has got its different tiers, then they're splitting sometimes with sub-tiers, and then you've got maybe one, maybe OpenAI's better for one thing.

[00:15:07] Dara: Even in our comparison the last episode when we were comparing Claude Code against against Codex. I had to think about that for a second, which is not a good sign for OpenAI. When we were comparing them, it was like Claude Code was overall better quality, but in some cases Codex was quicker, it was maybe cheaper, and maybe even more, so like from for what I was playing around with, it actually did a better job, but the code quality wasn't deemed to be as good. So there's so many different like axes that you could be comparing these different things on. It's getting complicated, and then when you introduce cost into that as well, which links to something we'll probably talk about in the news section in a minute.

[00:15:47] Dara: But this kind of like almost this kind of the, the cost of it is becoming more spread, isn't it? Where you've got the newer models are potentially becoming more expensive, but then you've got the cheaper models [00:16:00] are getting cheaper, but then it might be cheaper per token, but are you doing, are you using more tokens 'cause you're using more agents?

[00:16:06] Dara: And it's becoming complicated for people to know which model to use, which which f- w- not just even which, which model, but which, which model from which company should be used for what task. It's not straightforward.

[00:16:19] Matthew: Yeah. And on what power scale f- for each model as well.

[00:16:23] Dara: Yeah.

[00:16:24] Matthew: A- and I think that's part, like one of the things, so 5.6 is quicker and it scored higher on like the, I forget what benchmark this is. It's like the work benchmark as in do, just performing tasks and and sort of agentic capabilities, I think a co-work type thing. There's almost like there are some models, if you're using co-work, you would be better using Model X, or if you were using ChatGPT work, you'd be better using Model X and not this frontier model.

[00:16:53] Matthew: But if you're using code and it, it-- they're almost tailoring them to the different levels and use cases. And like you say, it [00:17:00] used to just be go in, pick top one, and ask question.

[00:17:05] Dara: Yeah. It's not so easy anymore. Yeah.

[00:17:07] Matthew: Nuanced now. Yeah.

[00:17:09] Dara: And that m- makes me think of something that actually we didn't wasn't on our list for the news, but I just thought of it now, and I haven't read much into it.

[00:17:15] Dara: I don't know if you have. I just saw a headline this morning about F- Fugu, I think it's called, F-U-G-U. It's a Japanese... I think the company is Sakana, and Fugu is the AI model, or I don't know if you'd even call it a mo- I think there is a model, but it's also like an orchestrator, so it can make decisions of which other models from other frontier AI companies it uses.

[00:17:38] Dara: And somebody on Medium was talking about it. I haven't seen anything else about it yet, but it seems like it's building up a bit of interest. Because what it made me think of is so you know when everything got really crazy around OpenClaw and all the kind of harnesses and everything, and that's all died down a little bit now.

[00:17:56] Dara: It does seem like there's kind of a space for some kind [00:18:00] of like orchestrator level to become a lot more popular now like even among like the wider beyond just power users even, where if companies are starting to look at like what models to use for what tasks, it's like there's a layer that's obviously not gonna be provided by Anthropic or OpenAI because they're gonna want you to use their models.

[00:18:20] Dara: But, almost like a, a space there for a gap in the market for somebody to come along and say, "Look, we, we'll plug into any of them, and we'll make sure you're using the right model for the right job."

[00:18:31] Matthew: And that, and that's-- What's his face has come out. I know his name, but I can never say it right.

[00:18:36] Matthew: CEO of,

[00:18:36] Dara: John Smith

[00:18:39] Matthew: Dario Amodei something like that, right? Yeah. Dario Amodei he's come out saying how dangerous open source models are because they're getting more and more powerful. And to your point right at the top of the podcast as the hardware gets better, the more that people could just run these [00:19:00] m-models by themselves.

[00:19:01] Matthew: And I saw an interesting take the other day that Apple might have played a blinder because they know that ultimately people are going to be running models on their phones and they're going to own the hardware, which will, which will service the open source LLM world at some point down the line.

[00:19:17] Matthew: If they just keep maintaining and building out the hardware that will be the vehicle for open source they're they're laughing.

[00:19:23] Dara: We called it.

[00:19:24] Matthew: Yeah. Yeah. Can't remember what we said.

[00:19:27] Dara: Can't remember what we said. Can't remember when we said it, but I'm vaguely sure we said Apple are gonna... Apple are a dark horse here.

[00:19:34] Dara: They're playing the long game. Yeah, I'll s- I'll agree with you. I don't know what we said, but yeah, I agree with you. I think we, yeah, we had-- I reckon they listened to that and decided to do what we said in that podcast,

[00:19:44] Dara: yeah. We're not sure what we said before, and also this hasn't actually happened yet, but we're very confident that we were right about a thing we may have said that hasn't yet happened.

[00:19:52] Matthew: Yeah. Yeah. But and the other thought, the only other wrinkle in that is if somebody comes out with a brand-new product [00:20:00] category that nobody can foresee, that, that sort of puts Apple on the back foot. But ultimately, currently, they own the majority share of the biggest hardware, what do you call it?

[00:20:12] Matthew: Category in the world. So they could just be sitting, waiting for the things to drop into their laps. People have kept trying to build this new hardware, but it could just all have been an app,

[00:20:21] Dara: i- I've got it right here.

[00:20:23] Matthew: Oh, yeah. The pencil's coming.

[00:20:25] Dara: It's the humble pencil. Yeah. Yeah, har- hardware is at risk of taking us off down another tangent.

[00:20:30] Dara: We did talk about this a goo- while- a good while back about saying the, the, the, the software is out, way outrunning the hardware, not just even in terms of the chips, but the actual... aI is not really, it's not g- we're not getting the best out of it by typing or even speaking into a phys- into a box.

[00:20:49] Dara: There's got to be some more what would you say? More application suitable. I don't know. I'm trying to think of a clever way of saying it, but, there's got to be a more kind of purpose-built piece of hardware [00:21:00] that takes full advantage of this. It's like we're trying to shoehorn a very new technology into something that we've been using for quite a long time, and it's not really, it's not really the right it's not th- there, it's not the right hardware at all.

[00:21:12] Matthew: No, and it is interesting that voice models have... Th- that's taken us some way there, but nowhere near as far as... i, and I think you're the same, I very rarely type as much as I used to. Sometimes I find myself typing and think, "What am I doing?" And I'm starting to get into that, I think 'cause I don't type as much anymore, I'm like tongue out, cl- clicking away with my index fingers.

[00:21:35] Matthew: But yeah though that's changed the way I interact with my computer, full stop. I don't even think about it anymore, but pretty m- it is a, a pretty seismic shift in how I interact with my computer. But yeah, like you say if you get into that point and you've got these frameworks and, what's the word I always forget?

[00:21:52] Matthew: Your open calls and things,

[00:21:54] Dara: Harness? ...

[00:21:55] Matthew: harnesses yeah, you don't-- And you can just talk to a [00:22:00] device and it have it go off and do things, and loop engineering where people are now saying, "Don't prompt."

[00:22:05] Dara: Yeah. "

[00:22:05] Matthew: Don't prompt your way through the solution. Prompt and orchestrate around it, let it do the rest," than the interaction with screens and things.

[00:22:13] Matthew: D- I think we're starting to go towards what we were talking about, we're gonna talk about in the later in the podcast, actually.

[00:22:19] Dara: Yeah. We j- we're just segue masters, but we're just... The problem is, the pro- the problem is the segue's come out too early.

[00:22:26] Matthew: Yeah. Yeah. Everyone remember that segue for later.

[00:22:30] Dara: Yeah.

[00:22:30] Dara: Yeah. Okay. So n- so news. So I've lost track completely.

[00:22:36] Matthew: Yeah, no, so that's OpenAI. That's what they've released, new models for. Interesting that it's aligning up with with Anthropic. Anthropic, I think like I said earlier with our all over the place news, I can't remember what we said in the last one, but Fable came back.

[00:22:51] Matthew: I don't know if we've touched on that. I think we probably haven't. So Fable came back, eventually was released by the government, the [00:23:00] American government specifically. And it's been back a week, and then they pushed it another week. So it, most people will probably remember that the, it was only ever gonna be temporarily available via the subscription model, at which point it was gonna drop off and just be an API thing.

[00:23:15] Matthew: But they kicked the can down the road to the 12th so this Sunday it's supposed to go from the subscription model again. They've been saying the whole time they want this to be a temporary thing and etc., etc. I don't know, I don't know if ChatGPT 5.6 might force them into keeping it around a bit longer.

[00:23:32] Matthew: I don't know. But yeah, it's been back for about a, a week and a bit now. And I think it's worth just briefly chatting about that whole debacle, 'cause we, I don't think we covered that either, the fact that the US government stepped in and said, "No, you can only use this for American citizens," which was a pretty terrifying precedent to set really.

[00:23:57] Matthew: It put me in mind, this might sound [00:24:00] dramatic, but it put me in mind of the after w- the end of World War II when there'd been this sort of international collaboration on the nuclear program and a lot of British scientists, and then the Americans told Churchill No, having any of the, the research or information, it's ours.

[00:24:17] Matthew: It's kind of that. It's such a powerful technology to be controlled by a single nation is not good. Y- yeah, and I, so if I got, if I seemed slightly distracted every second, it was, I was trying to just look something up that I'd read about it, which was that the interesting thing around it is how it was done as well.

[00:24:34] Dara: It was using export controls, which, so it's almost like there was no-- Like you used the word precedents, is because there was no way for them to actually block something like this. They had to use export controls, which is quite a, probably quite a novel way of... Somebody had that bright idea.

[00:24:49] Dara: Someone's probably got a pay rise for thinking of how to do it. But whether that will change if is this g- is this gonna be a is this gonna be a thing now and is this gonna happen more [00:25:00] regularly and are they gonna bake-- will they come up with some new legislation that actually lets them control this in a different way?

[00:25:06] Dara: And will that ultimately, will that hurt the US economy? 'Cause with China, making so much ground, is it gonna hurt, is it gonna hurt them? It's obviously a bit of a double-edged sword situation, isn't it? Where they're trying to- ... there's mult- there's like a, there's opposing motivations there, isn't there?

[00:25:20] Dara: As to whether they do control it or not.

[00:25:23] Matthew: I suppose it depends on-- 'Cause I've seen some way, I've seen some ideas of how they could control it, 'cause there's already even in the UK now with like adult content stuff there's ch- there's like checks and IDs and things that d- do you have to step through if you wanted to get into gambling sites or naughty things.

[00:25:42] Matthew: And like I saw some stuff in the US where they would maybe have to show their citizenship card and verify that to get access. 'Cause obviously just saying US US territory would just be bypassable via VPN. So that's how I get into most of Google's US-only features. So it, you [00:26:00] can see it being enforceable.

[00:26:01] Matthew: And as opposed to your other point- It's whe- it's whether they th- whether the government decides either it's too dangerous or if we isolate it to our nation, our productivity across the board in all other industries is gonna be so much heightened than everyone else's, then we could... That's how we, that's how it increased, that does them a favor to the American economy rather than the other way around of just an unbelievably exponential export to the rest of the world, but it makes me think and I don't like this line of thinking, but I can't see any way around it, that like the UK, for example, needs to invest heavily in getting their own controlled versions of this technology. Because if a nation can just turn it off overnight for you that's te- that's not good.

[00:26:53] Dara: Yeah. What happened to the it's just this isn't this wouldn't be a, at the national level, but within, within Europe, there [00:27:00] was a couple of good, interesting French, or I say a couple, I know there was one at least, it was like a front page-

[00:27:04] Matthew: Mistral.

[00:27:05] Dara: Yeah. I haven't heard anything about them since.

[00:27:07] Dara: I don't know if they're still-

[00:27:08] Matthew: They're still going. The last thing I saw about them was essentially a meme. They, they-- There was all these things saying "Mistral's released a new model," which was called like, oh, I couldn't find the the name of this Le Chat on Fat

[00:27:25] Matthew: So there's, there was all these rumors going around that Mistral had cr- created this Le Chaton Fat model that was far and away better than Fable, but it it turned out to just be a joke. But started by someone that just spread around the internet, and it was just all these pictures of a really fat cat and all these artificial benchmarks destroying Fable and stuff.

[00:27:48] Matthew: And then I think Mistral in the end got in- joined in with the joke, and they were posting stuff. So the-- all I've heard about them is that. But they still exist.

[00:27:56] Dara: Yeah.

[00:27:56] Matthew: Yeah. Yeah. 'Cause I was sure someone like Apple would gobble them up [00:28:00] or, but they're a good example of something, yeah, the European Union could push a lot of money behind to try and own something.

[00:28:08] Dara: Yeah, 'cause it is, it's dangerous. You're right, it's dangerous. If it's the US versus China and they both lock down, then, like where does everyone else get left if there's no technology outside of those two superpowers?

[00:28:20] Matthew: Maybe we'll just go back to skipping around in fields and daisies and then, and they can just destroy themselves in their little buckets, and we'll just get rid of our computers, touch grass.

[00:28:29] Dara: Sounds quite nice. I love your I love your idealistic view of the past, that we just all skipped around in fields with daisies.

[00:28:36] Matthew: That's right. There wasn't wars or anything, was there? I think that's just all we did.

[00:28:39] Dara: No. Just all happy times, rainbows and...

[00:28:42] Matthew: yeah. We yeah. It's mostly how I remember it.

[00:28:45] Dara: Yeah. So one, one other thing. Sorry, were you done there? I keep losing track. I don't even know what you were talking about.

[00:28:51] Matthew: Yeah, I think so. It's just th- those are my... I just wanted to vent my worries about that, that precedent of locking down the tech. I think I've [00:29:00] successfully vented.

[00:29:01] Dara: There was one other thing I was gonna mention, which we covered a little bit in what we were just talking about there, which is around this, So there's a couple of things prompted this for me. There was a, I think it was a CNBC article which was talking about how this, the idea of token maxing of companies being incentivize- or incentivizing their developers to use as many tokens as possible to, produce useful AI outputs that those days are coming to an end.

[00:29:24] Dara: And one clear example they gave was Uber, who used their entire annual AI budget in four months. And there's kind of a few articles floating around I've seen around this, and it's this- It goes back to what we were saying about the kind of the cost becoming, the, this, the maybe the workhorse models are getting cheaper and tokens are getting cheaper, but people are using more agents, so that's ramping up the cost even if the cost per token's getting cheaper.

[00:29:49] Dara: And then the new models that are coming out, they tend to be more expensive. Anthropic have shifted companies like, I think U- Uber were one of the first, weren't they? Where they moved them away from the-- They said you can't use the [00:30:00] subscription model, you have to pay as you go. So it's this broader-- I kinda wanted to bring up just this broader sense that the, the-- There's a few aspects to this, one is, are the AI companies coming under increasing pressure to be profitable, so they're having to not subsidize quite as much? And then with that, there's the trade-off maybe between investors getting impatient, but also that will the hardware catch up and will the costs actually be able to come down?

[00:30:26] Dara: And then the other point to do with all of that is the fact that companies are starting to keep an eye on this more now and be more conscious. Maybe there was a bit of a honeymoon period where companies were saying, "Just go and spend money because we need to try and invest in this," and now they're smartening up a little bit and thinking, "Actually, we need to look at what we, we are doing."

[00:30:44] Dara: And that links back to the point around all these different models and different options within each model and whether an orchestrator is the way to go. So a bit of a-- Rather than a news thing, more of a bit of a-- Seems to be a bit of a theme at the moment. There's a lot of talk around the cost of AI and the [00:31:00] different the different ways of using it and looking for efficiencies and also looking at the output and making sure there was...

[00:31:06] Dara: There was a stat I read, something like something like f- It's something like 40% of the cost that's spent is actually productive, something like that. This is really salty actually, but I'll, I might dig out the stat and share it in the show notes. But it's something like that. There's a point of diminishing returns basically, where if you spend more, you're not actually achieving more for that, that there's, it's really only like a small, a relatively small percentage of the usage actually goes to something that then becomes meaningful.

[00:31:34] Dara: There's a lot of wastage built in, and companies are becoming more and more mindful, as they obviously were going to. So a bit of a big sprawling point there. But it's something that seems to be getting a lot more attention now for a variety of reasons.

[00:31:49] Matthew: Yeah, hun- 100%. And I think I noticed it with Fable, like how much it can...

[00:31:53] Matthew: If that's so expensive, and we are on, we're on a subscription model, so we get allot- an allowance of Fable [00:32:00] reset on Sunday. I set it to task and told it to go and spin up agents. I stupidly didn't tell it to sp-specify what type of agents. I think it spun up a lot of Fable agents, and I used my f- five-hour Fable window in, I think, a minute and a half.

[00:32:20] Matthew: And then I, it came back up. I said, "Now, like use, use 4.8 and Sonnet 5 agents. Don't spin up agents for Fable." And I think within three minutes, it had used up my next five-hour window and my entire weekly usage. So it so what I'm I wanna go back and look at what that would've cost from an API perspective, 'cause I can imagine hundreds and hundreds of pounds for two or three minutes of of output.

[00:32:48] Dara: Yeah.

[00:32:49] Matthew: So it's like it, you've definitely gotta be, you've definitely gotta be cleverer. And I noticed like Anthropic is much more free with the amount of agents it can spin up, 'cause I've had it, I've had [00:33:00] instances where it's spun up 170 agents, for example. Whereas the new releases from GPT they're a, the agents it can spin up are like between four and 16.

[00:33:10] Matthew: Much more contained. I think there's a difference in the way they're architected. I think Claude does it as code, so it's not bloating the context, whereas Chat, ChatGPT's are literally spinning up in context agents. But anyway, yeah, slight side point, but yes, it, there's definitely gonna be more and more- techniques and approaches to stave off spending all that money on the most frontier models.

[00:33:37] Dara: Yeah. Maybe this leads in, maybe this actually, maybe this is a slight slightly more forced segue, but it does lead a little bit into our topic because I think it's, this could be an area that becomes an area of like focus for people within their roles. It could be something somebody manages where they're looking at, token optimization or looking at efficiency across using, and even just best fit models for different purposes.

[00:33:58] Dara: So yeah, [00:34:00] if you forgive me for that pretty rough and ready segue our topic today is around kind of human skill question and how the role of, probably for us, we're gonna talk a lot about analysts and engineers, but really it probably will apply more broadly across all, certainly all knowledge work in time.

[00:34:20] Dara: But how is that evolving with the kind of agentification of everything? So this is something that's come up. It's come up in bits and pieces, and actually it was also a topic a good while ago for an episode, which was before you and I were hosting the "MeasureBody." It was back when Dan and Bhav did, and they looked at the evolving role of the analyst.

[00:34:39] Dara: But it's something that comes up as a theme that kind of comes up a lot or has come up a lot recently about how roles in our industry, but really it's, it is a lot broader than that. It's how is AI changing the job, quote unquote. So for us, we are gonna obviously think about analytics, data analysis, engineering but it's, we're probably gonna [00:35:00] veer a little bit beyond that as well 'cause it's hard, isn't it?

[00:35:02] Dara: It's hard to talk about specific jobs when actually there's a lot of debate and discussion in society as a whole around how AI... And a lot of it's clickbaity and it's the usual polarized stuff. AI is gonna wipe out all jobs or AI is gonna make us all live in a "Star Trek" utopia. So there's a lot of stuff on either end, but I think if we can certainly start where we know the most, which is around our own industry.

[00:35:25] Dara: But I would imagine we will end up broadening it out a little bit too. And probably the catalyst maybe for doing another dedicated episode on it is around the kind of rise of of agents and the fact that things have... It does feel like there's been another step change. Probably when we last properly talked about this, it was around like it was still very much in the early days maybe of LLMs and it was pre-agent or pre- certainly before agents were being used quite as widely as they are now.

[00:35:53] Dara: So it's almost like a bit of a check-in, isn't it? It's a kind of a... And we'll probably cover this again. It's a bit of a kind of a, [00:36:00] almost like a snapshot of where are things at now? What are the skills that are most required? How's that changing? What do we think the future looks like?

[00:36:08] Matthew: Yeah, 100%.

[00:36:09] Matthew: I think it's something not just this subject, I think lots of subjects we're probably gonna have to keep coming back over and over again because, you used the term in the early days, which makes it sound like the '50s, but it was probably eight months ago. So yeah. And it's, I'm trying to think where to start really.

[00:36:24] Matthew: W- m-maybe we could start on a specific one of those two our world type things like the, from an, from the analysts analysis side of things. And then we've seen some pretty significant changes in visualization capability for one, right? Because we've traditionally it's like let's use Looker Studio to build a dashboard on top of whatever.

[00:36:50] Matthew: That's free and easy, or you've got your more enterprise level platforms. But with how good these models have got at spinning up [00:37:00] lightweight- HTML, JavaScripty pages and use JavaScript visualization libraries, you can get some really compelling, nice looking, dynamic dashboards and tell different data stories in a different way w- a lot quicker.

[00:37:18] Matthew: They look a lot more compelling, and they're so easy to produce that it's almost... It almost in my mind, moves you away a little bit from these, these static visualization engines where you're just plugging things in. So it's a different skill set. It's handing-- It's br- it's briefing the agent again to go and do the thing and then collaboratively working on the output.

[00:37:41] Dara: Yeah. It is, and it's not just the, that equally applies across the other kind of aspects of maybe what you would call the kind of like what has in the past been the kind of the hard, the hard yards of So yeah, building, you ask anyone who's built Looker Studio dashboards or Power BI or Tableau or whatever, it's painful.

[00:37:58] Dara: It's a painful process. [00:38:00] Nobody I'm gonna go out on a limb and say nobody really enjoys it. Maybe some people don't mind doing it, but I don't know that anyone has really enjoyed that. That's not why you become an analyst. You become an analyst to answer questions, to tell a story, to find areas for improvement or spot issues, and then figure out how to to resolve them, look for patterns and data, all that kind of stuff.

[00:38:19] Dara: But the-- but it's always been a quite a sinuous task, isn't it? Kind of wrangling data into a dashboard. And similarly with the other, whether you're a, like a SQL-type analyst and you're working in databases or even if just if you're somebody in a company where you work with spreadsheets and you create some charts based on that, whatever.

[00:38:37] Dara: Even that now, it's like you don't have to do anything in a spreadsheet anymore. The AI tools can all do that for you. They can write SQL for you. So e-every kind of... They can help you set up a database even where the line blurs between being an analyst and engineer. You can use AI to do some of the bits that you might have previously need to, needed to go to an engineer.

[00:38:55] Dara: So an analyst is almost can become an engineer, vice versa. So all [00:39:00] of that kind of heavy lifting is... The caveat here, obviously you've got to, there, there are issues and it's not perfect, and you do need to obviously know what you're asking for and how to check that what comes back is what you asked for and is useful and all the rest of it.

[00:39:13] Dara: But a lot of the stuff that would've taken, the real bulk of time before cleaning the data, sorting it out, getting it into whatever format you need, and then turning it into some kind of presentation layer or some kind of story, the time involved in that has just reduced down like almost to nothing.

[00:39:33] Dara: Minutes rather than hours or days or weeks. So with, with, with all of that kind of, don't really, maybe grunt work's not the right way to put it, but that kind of heavy lifting, all of that kind of the hard yards that frees up, and this I guess is the question, is it frees people up to do something else with their time.

[00:39:51] Dara: So what is that something else? So where do you, where does the human still add some value in that way? The human in the loop and what's the interplay between [00:40:00] how much can you offload versus what do you still need to do as the human analyst?

[00:40:04] Matthew: Yeah. A couple of points on that, c- I, it occurs to me that the grunt work is an ever-evolving, like more things are dropping into the grunt work category, where previously it would've, it would never have been considered grunt work, but just because of now the ease with which you have... The alternative can do it with such ease, it just becomes categorized as grunt work, where it used to may have been some deep skilled piece of something or other.

[00:40:30] Matthew: Yeah, that just popped into my head. But I think like the outputs of these things are great, and like you say, from an, from a, from an analyst point of view, delving into the data, understanding the data, finding patterns, building stories. What I see, like- What I see happening in, say, I see it, I see the, the job roles broadening out.

[00:40:50] Matthew: So let's stick with our industry for a second. I could see a world where an analyst, a data engineer, [00:41:00] a data scientist, a visualization expert all kind of become one and the same person, and it's just an individual who is very good at wrangling the tech, understanding the, the particular domain and working diligently a-and communicating that back out at the other end.

[00:41:21] Matthew: A-a-if they have those skills, they'll be able to deliver on a wider range of, a, a wider range of deliverables because they kinda have unlimited experts at their fingertips to point things at. And I think what I see happening with, say, like an analyst role but in the immediate term, is it maybe moves away f- it moves away a little bit from endless digging into data and instead moves into facilitating LLMs to dig into data which would be around semantics and around co- layering context on top of that data.

[00:41:58] Matthew: And I think that's [00:42:00] why, Google's put a lot of stock in their knowledge catalog and having all the semantics and metadata about data. That's why we've created and built SEAM internally. That's... I think that's gonna be such a huge point 'cause as soon as an, an LLM could understand what the hell's there- Then it becomes so much easier to produce compelling stories, compelling visualizations, compelling work

[00:42:25] Dara: I think you're right.

[00:42:25] Dara: It's like this k- kind of idea of almost like being a custodian of the definitions and a lot of that context. And I I wanna try my best to stay on point, but I'm at risk of just spinning it off. But I guess a question for you, just keen to get your take on this, is like I, I think in the here and now, it's like it's undeniable that there's real need for people who have the context, the context that the AI won't have yet and can't have, and a lot of that domain knowledge and the business knowledge.

[00:42:53] Dara: And you said the words Ian, when you were describing the skills that would be needed, like the, the communication aspect. I think that's more vital than [00:43:00] ever, isn't it? Because it's... And it's communication on two ends. It's taking the business need and translating that into something that you can put into a, into an LLM or an AI tool, and then also translating what comes back to the wider business.

[00:43:13] Dara: And that, that kind of has always been the real, if you boil it down, that kind of has been the core of being a good analyst, I think, in the past, and that's probably not gonna change. But it's the, the stuff around it and what you do to get that, from the input to the output. That's the bit that's maybe changing.

[00:43:30] Dara: But all of that kind of like context and the business definitions and the semantics and all the rest of it, Do you think that's a short-term-- Do you think once that's baked in will the AI technology get good enough that once there's enough, once there's enough business context across any individual business but also across businesses as a whole, is that just another area where it's gonna get smarter and smarter?

[00:43:56] Dara: And do you think it'll reach a point where that context doesn't come, need [00:44:00] to come from a human, or do you think there's always gonna be knowledge, context, nuance that's not gonna be able to be captured fully by a machine and would need some kind of human involvement? Big question.

[00:44:16] Matthew: Yeah. I think probably eventually, no, I think it does get there because, say, right now one of the big limitations of models is that they can't learn on the fly.

[00:44:28] Matthew: So there are gonna be, like you say, nuances and differences and quirks to, to s- two same businesses in the same domain. They're gonna be slightly different, work slightly different, have different approaches, maybe slightly different offerings. So there's-- It won't just be able to go in with one knowledge of one domain and away it goes.

[00:44:47] Matthew: But as soon as the-- A-and this is a problem I know that is being actively pursued right now. As soon as they're able to, it's able to update its thinking and tailor itself and almost learn on the fly, then it's a, [00:45:00] it's a solved problem then because you can put it into a business, have it start to explore and find areas and feed that back into itself, and it just becomes a, it buil- it makes itself a domain expert.

[00:45:12] Matthew: But how far down the line that is I don't know. And I think the other limiting factor here that, that slows any of this down is the pace It's the pace of differing ind- industries. Some are gonna be on it, and they're gonna be moving quickly, and they're gonna be adopting this stuff quicker. Uber is probably a good example, a, a essentially a technology company a disruptor in their own right originally, and they've, they jumped on this quickly.

[00:45:37] Matthew: But then you've got industries like education or healthcare and healthcare administration anyway, but things like that, that are just going to take a long time to adopt this stuff. And job roles may stay more static in those places than they do in wider industries. And it may be that [00:46:00] the only...

[00:46:00] Matthew: you can imagine whole industries essentially being complicit with plodding along. Everyone in that industry is like, "Just, we'll just stay here." And what may kick them into gear is disruptors entering in because everyone can build things, everyone can create amazing stuff. A disruptor appears, and then all of a sudden everyone's gonna be forced into action.

[00:46:20] Matthew: So yeah, I think that's the main limiting factor in, in it going where you're describing, like that eventual They can just do whatever they want. Agents can just take over.

[00:46:32] Dara: I think you're right, and I think the other thing I think that m-maybe sits alongside that is even if you have a very progressive industry or company, and so let's say take Uber even, they might be fully bought into using the AI, and they might be using agents for a lot, but there's still that but I'm gonna call it political, but maybe that's not quite the right word, but there's that there's the people aspect and if different people in different departments, if it's a [00:47:00] really forward-thinking, AI-savvy company, then you'll have people in finance using AI, you'll have people in marketing using AI, you'll have people in product using AI, but they're still gonna be coming up with different recommend...

[00:47:11] Dara: Even if they're using AI to fully tell them, to plan, to strategize, to tell them what they should do, there's still gonna be that human layer where finance says no, we've done our research and we think we should do X," and marketing say we think we should do Y." And that's probably still where there's a role, again, sticking to where we started, which is around kind of the analyst role.

[00:47:31] Dara: I still see a need for a human who, even if their day-to-day, what they're working on, what, and what hard skills they need to know, there's a bit that doesn't go away, which is that ability to communicate and mediate and present a a, an objective view of what the, the facts-based data is, and whether it's outputted from a LLM or whether it's from human effort and going through and looking over different inputs, it doesn't really matter.

[00:47:57] Dara: There's a need for somebody who can pull those different [00:48:00] pieces together and give a, an objective view of what the right course of action is.

[00:48:05] Matthew: Yeah, potentially. I think there's two... What, there'd be less of them in that scenario, for sure. And I think to, to go back to that point earlier about the sort of technology wrangler, communicator, I think there's, within any, any sector or job role, there are people who are brilliant analysts who can delve deep and absolutely their skill and talent is in the, the technology or the knowledge of going through a piece of analysis, the skill and the understanding of it.

[00:48:40] Matthew: I think unless they have the other things, they may begin to struggle more, and I think it's the people who y- they could just be picked up and put on top of any of the other and it could be put on top of engineering or visualization, and if they had time to learn that or if that was their chosen field in the first place, they'd shine in that.

[00:48:57] Matthew: I think those are the people that will ultimately bubble to the top [00:49:00] because they could then orchestrate across the... So I think it gets smaller. There's less people. So in that scenario where you're saying, there might need to be an orchestrator, there might need to be an arbiter of whatever it is, yeah it shrinks.

[00:49:13] Matthew: But then I would also say There's a lot of experiments going on of these virtual societies and virtual organizations where there's an orchestrator agent. There's a-- They have different personas and roles, like a marketing team and multiple people within them, agents within the marketing team, or there's a CEO, and these different structures where you can imagine them getting smart enough that's...

[00:49:41] Matthew: The, the output is compelling. G- I guess what you... Guess what your what your, what you're saying there relies on there being some divine spark or something special about how we make decisions or come to conclusions.

[00:49:54] Dara: Y- yeah, which is a yeah. That's a, that's an interesting observation 'cause it [00:50:00] is something I've been thinking about in preparation for this.

[00:50:02] Dara: One thing just before we go onto that. So it's interesting what you were saying about the first part, 'cause I agree with you that potentially, and this is where it doesn't just apply to analysts. It, there's a logical kind of train of thought, isn't there? That like if, as the AI gets better, just in general, there should, in theory, be a need for less specialist people to do these specialist roles when the AI can do that, and you might still need humans ultimately making the decisions.

[00:50:26] Dara: But in theory, there should be a need for fewer of them because more of that groundwork is being done in an automated agentic way. But maybe there's a kind of a slight bit of a flip side to that where you were saying so there might be fewer analysts, but you might also argue that more people would become what you could call an analyst.

[00:50:47] Dara: So somebody who is, who comes from a finance background, maybe you'd argue they're an analyst anyway, they're a financial analyst, somebody who may be a lawyer or something, they are then an analyst as well. So-

[00:50:59] Matthew: There's the [00:51:00] idea of Je- Jevons paradox. Have you come across that?

[00:51:05] Dara: Remind... Yeah, I've heard it, but remind me what it is.

[00:51:08] Matthew: It's a guy, it's a guy, it was around about the, I don't know, the Industrial Revolution and coal usage was going up, and then someone invented a much more efficient steam engine, and then he, everyone was predicting the decline of coal use, and he predicted that wouldn't be the case.

[00:51:29] Matthew: So the paradox is I don't think it's actually a paradox, but it's what he calls it. What a- what actually happened is all of these new use cases started to appear for coal when it's cheaper and, sorry, when the, the steam engine is cheaper and more efficient. So railway came out of out, out of it, and we started to transport things via railways, and factories started to bring in more.

[00:51:53] Matthew: So actually coal usage- Increased when efficiency increased, which is paradoxical. [00:52:00] So there's the idea, a, a similar idea in AI that, yeah, maybe everyone just starts covering more hats, but more work comes in. So the same, the number of individuals remains, but they just cover a wider area because they, because they can, and we just Capability increases, and then the work just fills to, just expands to fill that new volume is the other theory.

[00:52:24] Dara: Yeah, and that's what's happened a- at every point with every revolution or every new big technological advancement. That is what's happened to date. So you could ar- that's one school of thought, isn't it? That like there's no reason why this wouldn't be any difference different. And maybe it's like a blurring of...

[00:52:40] Dara: And I guess that's where I was trying to go with the point about maybe the number of, quote-unquote, "analysts" might reduce, but the number of people who have those same skills might increase. It's like maybe there'll be like a blurring of roles where people are s- are, maybe they're not specialists in lots of things, but they have the ability to be specialists because they're [00:53:00] augmenting themselves with AI.

[00:53:01] Dara: So they may still have their specific domain area or scope or whatever, but they're also able to broaden that out to a lot of other areas because they're augmented by AI.

[00:53:14] Matthew: Yeah. And I think that is the next logical step, where you almost just have an individual in a, with a wider view of say, just a data person as a data wrangler and a, a people wrangler or whatever.

[00:53:31] Matthew: And then, we've talked about it internally at Measurelab, almost an organizational tree that underneath those individuals are the different agents and personas that service that section of the business. So underneath the data wrangler, there might be an analyst, and there might be a data visualization, and there might be a data engineer, and there might be a implementation expert.

[00:53:51] Matthew: And ultimately, this person is wrangling and orchestrating those people, 'cause it might be a bridge too far just to have an in- it's probably possible, but for one individual to [00:54:00] cover absolutely everything and wrangle all of the different sections and personas of all of these different agents across the board.

[00:54:06] Matthew: I think it might get there, but for now, the next logical step feels like just much tighter integration with agents and people in an org.

[00:54:15] Dara: Yeah. Yeah, quite possibly. And it, the whole thing might just drive capitalism into overdrive because it might be that small companies can be big companies, and big companies can be huge companies, and huge companies can be absolutely behemoth companies.

[00:54:29] Dara: And everybody might just produce more. There just might be more and more output.

[00:54:34] Matthew: We all need really to be striving to, get Elon Musk back over that trillion just to make sure, that's what we should all be striving for. Make those people really rich.

[00:54:44] Dara: Yeah.

[00:54:44] Matthew: Yeah. Yeah.

[00:54:45] Dara: Yeah. '

[00:54:45] Matthew: Cause if he's dropped back to 800 million, billion or something is- It's sad ... it's really sad. I hope he's okay.

[00:54:51] Dara: It's just not a big enough number, is it?

[00:54:54] Matthew: No.

[00:54:54] Dara: But it is, like this is the, this is the-- And this has come up a couple of times and maybe in slightly different contexts, but this idea [00:55:00] of there's a, the belief that we're gonna free up all this time, but actually, evidence in history does suggest that the more productive society gets, the more productive they seek to become, and it's just a never-ending.

[00:55:15] Dara: So-- And we found that too, like n equals one kind of experiments, but with our own, when we-- on our own journeys where we've been going through exploring all the different kind of uses of AI and like trying all these different things out. And Mark Edmundson, I think he might have been the first one, the first guest we had that talked about this, where, you're creating all this extra time, but then you have this urge to use that extra time even more productively.

[00:55:39] Dara: So you might, you're becoming more productive, but it doesn't necessarily mean you're freeing up... you are freeing up more time, but then you're u- doing even more with that free time, and maybe that's part of the kind of human condition where even if we become more efficient, we'll just find more things to do with those efficiencies to become even more productive.

[00:55:59] Matthew: Yeah, [00:56:00] potentially. I think the only thing that gives me pause on compare, comparing this revolution, which I think at this point is probably undeniable that it's a, a another industrial revolution or a or alike to the others is that the fundamental component that we're talking about here is intelligence, and that is so different to anything.

[00:56:20] Matthew: Previously it's been a, a resource or a, a new technology, but we're literally talking about the fundamental thing that has progressed us as a species over any other on the planet. It's it's so hard to quantify what that looks like, and there's literally an unknowable point by which, you know, when you reach AGI or beyond, we can't know because we don't have the capacity to be able to view past that point.

[00:56:44] Matthew: So it, yeah, as, as always, I'm always just, I always just come back to like it's intelligence. This is, the resource is intelligence, so God knows if anything, paradox or precedent matters, it could just be all moved because of what we're actually [00:57:00] dealing with.

[00:57:00] Dara: I was-- It's interesting 'cause I was thinking along the same lines where, because that's what a lot of you'll have done the same.

[00:57:07] Dara: I've heard people on podcasts I've read stuff that people have written, and people who are really in the know, people who are like right at the front edge of this stuff, and they've compared back against the other big, the electricity, the internet the printing press, all of this, and they're basically saying, "Look, this is no, it's no different.

[00:57:23] Dara: Each time around, people have worried, said this is gonna wipe out jobs, it's gonna, ruin society," whatever. And maybe there's been a period each time where things have been, because they have, like each, each time around a lot of jobs are lost, but then new jobs get created that didn't exist before, and things find a way of, maybe not everyone would agree they balance out, to, on a wider level, they balance out.

[00:57:45] Dara: But the difference like you were saying, the difference this time around, at least potentially, is that the th- so before what the new technology did was let people, humans, focus on that higher level work. So they abstracted themselves and [00:58:00] thought instead of like hand weaving, I'm now gonna supervise machines that hand weave," or, "I'm now gonna design machines that hand weave."

[00:58:07] Dara: There was like a level above that you, that we could move ourselves to that was like a cognitive level. But with AI, it's actually chasing the same step up above that we would want to move into. So if that gets cracked, that if AGI, and you're right, like we can't really, none of us can really fully wrap our head around what that would mean, but potentially that is gonna fill that step that we would have otherwise moved into.

[00:58:33] Matthew: Yeah, 100%. And I think a lot of these, what annoys me about a lot of takes in these podcasts and alike is they're all very binary, right? It's yeah, but in previous industrial revolutions, new jobs that we've never thought of or even a- anyone ever conceived of have been created.

[00:58:51] Matthew: Okay. But that will still happen. There will still be new jobs created that nobody has ever thought of. It doesn't necessarily mean that [00:59:00] there's a, th- there's like a little pile of new jobs and then a giant tidal wave coming over the top of it. The two... Just because new jobs will be created doesn't mean problem solved.

[00:59:09] Matthew: It just means new jobs that we haven't thought of will be created. So I think, yeah it can be too, it can be way too binary and way too like black and white.

[00:59:17] Dara: Like with everything.

[00:59:19] Matthew: Yeah. Yeah. A- and you're right, like the higher order stuff that we've been moving up into is vanishing.

[00:59:26] Matthew: So much so that it's more, people are viewing moving back to, into the physical world as the step. Like that's our remaining domain is physical, become a plumber, become a woodworker. Robotics is advancing pretty quickly now, but-

[00:59:39] Dara: Yeah. Yeah ...

[00:59:40] Matthew: it won't be forever, but that's the sort of temporary measure.

[00:59:44] Matthew: Start, start f- plowing fields. You do get a bit of hope when you watch. Did you see that video that went around a little while ago that was like the ro- in China, they have robots running marathons and half marathons alongside humans, and there was like a compilation video of all these like robots falling flat on their faces.

[00:59:59] Dara: But it only gives [01:00:00] you brief hope because it's not gonna be long before that advances as well. So you're right, the physical world isn't necessarily gonna be a safe haven.

[01:00:08] Matthew: No. And the advance... The, the, I, I've watched, I've been watching similar videos where it's, it looks at those kind of robots that you just described, and then robots of 2026 And even the improvements is-

[01:00:21] Dara: Yes

[01:00:22] Matthew: unbelievable. I think a- and the tech is getting cheaper, the motors, the dynamos, the everything is just getting so cheaper to produce on that side. There's definitely some pretty interesting things coming from robotics. I think I do think like the dexterity of laying down under a sink and trying to get around something and do whatever is gonna, it, it's gotta be a long way off, surely.

[01:00:44] Matthew: So I think any, any plumbers and electricians listening, they're probably-

[01:00:49] Dara: Y- yes and no, 'cause you think are we really purpose-built for that job? You can't think we are. There's got to be-

[01:00:55] Matthew: You have a sink com- a sink robot that just does that exact thing.

[01:00:59] Dara: Yeah, it's just [01:01:00] specialized, like it's the only thing it can do.

[01:01:03] Matthew: Yeah. A massive van pulls up and then there's all these different robots for the different parts of the house drive out.

[01:01:10] Dara: So go, going back to so pulling this back to what we were, we- to where we started a little bit. So yeah, and something you said earlier about what, like kind of what is this, this kind of sp- unique spark that we have a- as humans.

[01:01:21] Dara: So if we take the-- Well let's not give our- let's not restrict ourselves. We could talk about analysts, or we could talk about anybody in any knowledge working job. What's the bit what's the bit that we still, what are the bits that we still add? So we're all augmented, we're using agents, we don't have to do a lot of the toil that we had to do before, whether we call it grunt work or not.

[01:01:43] Dara: What are we what's safe? What, what skills or characteristics or competencies can we focus on or feel safe that we have, that we can still add value in addition to having some of the, these kind of tasks and this the tactical stuff done by AI? [01:02:00]

[01:02:00] Matthew: Now?

[01:02:01] Dara: I thought you were just gonna say nothing.

[01:02:03] Dara: We don't.

[01:02:04] Matthew: No, I think, I th- that's my, that's my question there. Right now, I could probably list some things. Down the line, I d- I don't know that I do. Nothing obvious jumps out to me as like we have that, that couldn't be replicated. But now I think our ability and efficiency in the way we can collaborate and communicate certain things and concepts is higher.

[01:02:28] Matthew: Our ability to absorb and tailor to an individual domain is there. All these models are just giant black boxes of everything, but we're, we can be more nuanced and expert in smaller fields. What do you think?

[01:02:44] Dara: Yeah no I liked where you, how you split it in two because that's where my head was going as well.

[01:02:48] Dara: It's like what is there now? And this is a really, it's a really philosophical question, isn't it? Because I think you, when you, I'm sure you did the same. I d- went and did some actual reading for this, but then I also just had a chat [01:03:00] with Claude. And the stuff that comes back immediately, it's like things like taste, it's decision-making, it's judgment.

[01:03:06] Dara: It's all of these kind of like things that we do hold dear and think, "Oh, that needs a, it need, it needs a human." And obviously then there's things that, that kind of go more into there may be not specific things like that, but there's obviously the a bit like p- people at the moment, the way things are currently, people I think give more stock to...

[01:03:25] Dara: if you told me something, it depends what it was, I might think it was salty, but if you told me something, I'd give that more weight to maybe something that I felt might have been hallucinated by an AI. But even that will change. So there's things at the moment where we like human contact, we like knowing that we're hearing from someone from their own lived experience versus something that's just come from the big soup pot of, human text that is AI.

[01:03:50] Matthew: I think there's an irony in what you're saying. I think under- underneath what you're saying, what humans remain good at is non-data-driven [01:04:00] decision-making.

[01:04:01] Dara: Yeah. Which is that it... Is that even, is that something to be good at? Is does that, is that a bit of a...

[01:04:07] Matthew: But it is b- the, the say, say in the analytics industry, for years we've been fighting against C, C level or higher up folks in an industry just going with their gut and making decisions in that way and saying, "No, data.

[01:04:18] Matthew: Use data, measure, make informed data-driven decisions and that will improve things." If there's data to back up decisions, then ul- ultimately an LLM is gonna be good at spotting the pattern and making a recommendation. So what's left is the n- is the non-data-driven decision-making where you just go, "Yeah, I reckon this," or, "I trust somebody more because of this."

[01:04:38] Dara: I gave you one job before the podcast, which was to stop me going dystopian, and I try my very best, and then you actually do the opposite and you try and just, steer me into the, steer me straight into the void. I'm I was b- almost playing devil's advocate there. I was thinking like, what is...

[01:04:54] Dara: 'Cause it, it is hard to avoid, and this is where we are gonna go. I'm like, "Do you know what? I'm not even gonna [01:05:00] apologize." This is worth thinking about, it's worth talking about, and everybody should be thinking about this, so I'm not gonna apologize for going dystopian. But it's h- it's hard to not go to that logical conclusion, isn't it?

[01:05:10] Dara: 'Cause you, it does feel a little bit like clutching at straws. It's like, "Oh, we can do this thing." But most of those things that you can think of- AI will be able to do them in time.

[01:05:22] Matthew: It's pace. It's just pace with which we move through it, it feels to me like. Pace with which we move through it and the different industries and how they quickly they move through things.

[01:05:31] Matthew: Ultimately if things keep improving on this exponential curve which they have been doing for a number of years. We had a blip last year. Everyone's like it's over," and then like Fable comes out and blows everyone's heads off. It's like clearly not yet. There still seems to be a, a more compute, more inference, higher exponential output.

[01:05:52] Matthew: If that trend continues, and there-- I, I don't see how there can be any other conclusion to I don't know, I don't know how to finish that sentence. To, to think [01:06:00] more and more things being taken over by LLMs. The only other thing we've got going for us right now to, back to your token point is, in some instances, we're probably cheaper.

[01:06:09] Dara: Yeah.

[01:06:10] Matthew: A dev is probably cheaper over the course of a year.

[01:06:13] Dara: Yeah.

[01:06:14] Matthew: But interestingly, not for the same output.

[01:06:16] Dara: No.

[01:06:17] Matthew: But like you said, you fill the void with the work you do, and that costs a lot more money. It's, it is, it's a minefield.

[01:06:24] Dara: It is a minefield. What about, so like often the talk is around j- like judging the output maybe a- and again, at the moment, so let's just say forget about where it goes in the future right now.

[01:06:35] Dara: There's a need to verify what comes out. There's a need to like then maybe tailor that into a message for other humans or whatever. But what about the input? Do you think the, do you think the AI can get to a point where it can also tell us what we want in the first place?

[01:06:50] Dara: 'Cause say it gets really good at, if you've got enough data, it can understand that data, and it can tell you what to do to achieve the goal that you want to achieve. [01:07:00] Can it also get to a point where it can tell us what the goal should be in the first place?

[01:07:05] Matthew: I think so. If it can get to, if it can start to do that sort of absorbing and understanding and self-learning of domains, then it can yeah, it can start to put together the pieces of 'cause ultimately, any goal it makes would be a sub-goal. E- every organization roughly has our goal, right? They're trying to achieve our thing. That's the very point of an organization even existing. If, and that can be like a business organization, it can be a societal organization. They all have some reason for being, some, some ultimate overarching goal.

[01:07:36] Matthew: I think an LLM w- creating sub-goals underneath that to achieve the ultimate goal is- It's possible.

[01:07:45] Dara: Scary. I'm just trying to... I'll try... is this episode going where we hoped it would? Is it going where we thought it would?

[01:07:52] Matthew: I don't know. I think, yeah we started off brighter, but then I...

[01:07:57] Matthew: And I think this is reflective of the subject, [01:08:00] right? The longer you think about it, and the deeper you delve, and the more you scratch, you just keep adding answers to that, and it-

[01:08:05] Dara: Yeah ...

[01:08:06] Matthew: inevitably it's hard to... There's no one big obvious block that's "They aren't getting past that. LLMs aren't smashing through that human, like I said, divine spark.

[01:08:18] Matthew: That, that is ours forever." I think people made that mistake with the creativity stuff and the ability for it to produce images and music and all those sorts of things. And listen, I'm no arbiter of art quality and I'm I can appreciate that people will say it isn't creativity," or whatever, but people thought that it would never get anywhere near that.

[01:08:40] Matthew: It's not gonna be able to produce a picture or something that looks like this or could win an art contest that's ours. That's our special little domain. But then it did. First. We did that first.

[01:08:54] Dara: Yeah. Yeah. Yeah. Yeah.

[01:08:57] Matthew: So yeah, I think that I think it is what [01:09:00] it is.

[01:09:01] Dara: It-- look it is, and the n- no one...

[01:09:03] Dara: Two things are absolutely true. One is that nobody knows for sure. There's a lot of y- and you can talk to, and even people we've had on and like we've only had even if you take the guests that we've had on, and we've had people's takes on this, like they've even barely scratched the surface of all the different kind of thinking there is around all this.

[01:09:21] Dara: And the and one thing for certain is nobody knows for sure because we don't have, like we, all we can judge it on is how it's improved from where it was before to where it is today, and we don't-- It may stall. There's so many b- it could stall and then we'll spend years and years making the most of what we already have, 'cause we've made that point before where even if the advancements stopped tomorrow, there'd still be huge gains to be had from what it can already do that people aren't taking advantage of yet.

[01:09:46] Dara: And

[01:09:47] Matthew: we said that, the point at which we said that compared to technology now is hilarious. It, the, it's moved on so much since then that the, what's left on the table is s- like smoke sitting on [01:10:00] a giant pile of gold. It's it's it's mad how much is, how much opportunity is still there, and it's, and that, it's like it just keeps getting bigger, and we're not scratching, nobody's able to consume it all.

[01:10:10] Dara: Yeah. It's so anyway, so yeah, so we don't, so no one knows for sure. And then the other thing is the pace, so even if it does, even if things do change beyond recognition we don't know over what time period that's gonna happen because there are gonna it's gonna, it's not gonna be, I don't think it's gonna be a switch necessarily that just one day we wake up and realize that overnight everything is completely and, irreversibly changed.

[01:10:34] Dara: It's gonna be a, it's gonna be a phase thing, and whether that phasing happens over a year or a decade or a century or longer we don't know for certain, do we? But I think we we certainly agree, and I think probably a lot of people, maybe a lot of people who listen to this podcast or a lot of people who are within the, either within within AI specifically or just within tech and, people who even are just who have an interest in this I don't think anyone would disagree that this [01:11:00] is a big turning point and, it is gonna be a point where people look back and go, "That was a, that was a pivotal moment."

[01:11:08] Dara: I don't think anybody could deny that.

[01:11:10] Matthew: Not anymore. There was, maybe a, a year or so ago there was those detractors but they, their voices have quietened down a fair amount now. There's still detractors in various forms that, that focus very much-- they always focus on the now, like the cost of this or the, or whatever.

[01:11:28] Matthew: But they're only temp- the, the temporary hurdles that it's easy to see how they're overcome. This intelligence stuff will get cheaper and I think it's it's all about, it's all about a couple of different factors to how quick it moves. If the technology comes, becomes cheap enough and it's u- and it's sort of orchestration becomes easy enough, then I can see things moving quickly because individuals within organizations can cheaply and efficiently start to deploy it to complete meaningful [01:12:00] work.

[01:12:00] Matthew: At that point, it's like the the stakeholders, the, the board members take over and it's like what is the cheapest and most efficient way of us delivering on this thing?" And that has always won out, unfortunately, within capitalism and That's how I could see it rapidly moving over a couple of years rather than decades.

[01:12:19] Dara: But there's an offset against that as well. So if that's the way corporations go, there's also then the flip side, which is individuals can almost be corporations or look like them overnight. So there's gonna be a-- E-e-e-even if it does go that way, where it's like everybody can do everything, then it's, there's gonna be a bit of a battle between existing big companies who maybe start thinking, "Okay we can achieve the same productivity and the same profitability with far less staff."

[01:12:47] Dara: But, on the flip side of that, you're gonna have a lot of people thinking, "I could basically be Salesforce tomorrow," or, "I could be," maybe not Google, "Yeah, I could be a big software company tomorrow if I want to be." And go back to your point that you [01:13:00] made earlier about some of the slower industries and, disruption might be the thing that actually spurs them into action.

[01:13:07] Dara: That there's tons of opportunities out there now for that. Anybody with a bit of entrepreneurialism could go out and disrupt an industry today or tomorrow. I like to think we're inspiring this. We're gonna-- This podcast episode is gonna be a turning point as well. We're gonna have a load of people go out and disrupt industries as a result of this.

[01:13:24] Matthew: Yeah. Go out and take down the-- Everyone just strive to take down Google. Build a new search engine. Sam Altman said a few years ago, didn't he? I, I keep saying a few years. I've so-- One thing this has all done is it's warped my sense of time so much. Like I- He

[01:13:39] Dara: wasn't alive a few years ago, was he?

[01:13:41] Matthew: No, I don't think I was.

[01:13:42] Dara: He's only three or four years old.

[01:13:45] Matthew: But he said there will be, there will be the first individual company billionaire at some point, which at the time I think was chuckled at a little bit, but now doesn't seem crazy at all. Seems it seems inevitable and in short order probably.

[01:13:59] Matthew: [01:14:00] So maybe they do know what they're talking about. But I think, like we said, like it's-- all of this is unknowable. This is just too- Two clowns on a podcast who are pr- trying to predict what the future look like, looks like. It could be completely different and in a complete different direction.

[01:14:18] Matthew: One thing that is true is it's, for probably for most of the people listening to this podcast, because there's probably a f- a high nerd percentage and a high nerd saturation in the audience, it is incredibly exciting and engaging. I have, I'm more excited I've been more excited and engaged over the past three years than I have been at any other point in my career.

[01:14:45] Matthew: It really feels like you're living through a piece of history and you're part of it, and you're, even if you're a little piece in the cog, you're your machinations and your experiments and your discoveries are building into a whole. That's positive, right? That's, [01:15:00] we're in history, whatever the outcome is.

[01:15:03] Dara: Absolutely, and there's no changing that, so we might as well lean into it. And I guess if there's a piece of advice for anyone listening, and that's okay, people listening, but also you'd love to get the message out to people not listening, which is make sure you're using these no matter what job you work in.

[01:15:18] Dara: Because this whole thing And just to link back to another point made on the podcast before, the fact that, you know the point around how I don't know if they actually said it or not, but this belief that Anthropic went after code first because it's the, the foundational layer, and then once they got that sorted, basically a- anything and everything you could do on a computer was fair game.

[01:15:38] Dara: So if you're-- and that's most jobs these days, so if you're anybody who works with a computer, which is most people. Yeah. Yeah. Which is so most people, it's scary, it's exciting, it's uncertain, but you can't change what the outcome's gonna be. But what you can do is lean into this stuff and play around with it, and that's exactly what we did.

[01:15:56] Dara: You've been doing it for a particularly long [01:16:00] time. I was a bit slower to get fired up about it, all of our own kind of individual journeys, playing around with lots of different things, testing things, breaking things, a lot of them failed. Building little bits of software, building personal things, whether it's a little like thing for managing your personal finances or for me, building little running dashboards and things like that.

[01:16:20] Dara: Just playing around with this stuff and learning the skills, learning what they can do and what they can't do is never gonna be a waste of time because every job is gonna benefit from having this knowledge and having these skills. And where it ends up, nobody knows. But in, there is gonna be a medium term where the more you know about how to use these tools, the better you're gonna do in your job and the safer you're gonna be.

[01:16:43] Dara: And the, the bigger changes that happen, they're outside of any of our control, but all we can control is making sure that we're using the technology, we understand it, we're using it safely and sensibly, and we're keeping up to date with what's changing, which is not easy, as much as possible, keeping up with the news it's probably, arguably it's [01:17:00] the most important thing to be keeping up to speed on at the moment.

[01:17:04] Matthew: Yeah, everything else is bloody depressing, so at least you can get involved with this and put a positive spin on it. And I think the other thing, like one thing I've learned You know, doing it for a few years is not to be precious or disheartened with failed experiments or ideas that don't come to fruition.

[01:17:22] Matthew: They will-- If you can do things in a much more, in a much quicker pace and much more exponentially, there's gonna be a bigger pile of failures and things you leave behind, and you got, you, you gotta not be precious and just keep going and keep moving and keep experimenting, 'cause eventually you will come to something that's "Yeah, I've nailed that.

[01:17:38] Matthew: That's great. That works for these two months."

[01:17:41] Dara: Yeah.

[01:17:42] Matthew: Yeah.

[01:17:42] Dara: All right. Cheery cheery but necessary. No, look, it is what it is. We shouldn't... I think this does get, these conversations do get heavy but it's something that's on people's minds, and I don't think it's, I think it's it is a conversation that's worth having regularly because, it's something that is affecting [01:18:00] everybody.

[01:18:00] Dara: So it's not something we should shy away from. But it is, you do feel a little bit like you're almost being a bit doom and gloom, but I think both of us, we have a mixture of positivity about the opportunity this is providing, but with a sensible amount of fear and uncertainty because it is changing at a rapid pace, and nobody does know exactly where it's gonna end up.

[01:18:19] Matthew: Yeah. 100%.

[01:18:21] Dara: All right. Look forward to the next time we do this and we're floating around like they do in "WALL-E," where we're just in these little floating beds.

[01:18:29] Matthew: I'm nearly there. I don't leave this chair.

[01:18:31] Dara: That's true, actually. Yeah. Okay. All right. That's a wrap for now. See you next time.

[01:18:37] Dara: That's it for this week's episode of "The Measure Pod." We hope you enjoyed it and picked up something useful along the way. If you haven't already, make sure to subscribe on whatever platform you're listening on so you don't miss future episodes.

[01:18:48] Matthew: And if you're enjoying the show,

[01:18:49] Dara: we'd really appreciate it if you left us a quick review.

[01:18:52] Dara: It really helps more people discover the pod and keeps us motivated to bring back more. So thanks for listening, and we'll catch you next [01:19:00] time.