#126 First-party data strategy – a client side perspective (with Roman Petrochenkov at Carwow)

[00:00:00] Dara: Hello and welcome back to the Measure Pod. I’m Dara, joined by Matthew. Hello, Matthew. How are you doing today? I’m good. 

[00:00:21] Matt: Yeah, that was more abrupt than normal and I’m, I’m all about efficiency, straight to the point. Yeah, well I normally prevaricate trying to figure out what to call me for, for sort of five minutes or so, maybe straight, straight in.

[00:00:31] Dara: I’ve been doing some self-analysis and by self-analysis, I mean I’ve been getting, chat. GPT, shouldn’t promote one. I’ve been using an LLM that shall remain nameless. Others are available. Two, yeah, others are available. They may or may not be not as good. To analyze me and this podcast, and it told me, I mean, I’m doing it now.

[00:00:54] Dara: It told me I’m, I’m long-winded and slow to get to the point. So I’ve decided to take the feedback on board, so I’m going to be Mr. Efficiency from now on. 

[00:01:01] Matt: It’s part, it’s part of your charm, isn’t it? A bit of ramble. A bit of ramble. 

[00:01:05] Dara: A suitable, a bit of, yeah, exactly. I’ve gotta find that sweet spot. Yeah, exactly. Yeah.

[00:01:12] Matt: How are you?

[00:01:13] Dara: I’m, I’m, I’m all right. I’m, as is the wonder of podcasting technology. We’re basically time travelers. so I think by the time this is in your ears, listeners, I’ll be in Spain, not on holiday. 

[00:01:29] Dara: No, no. Moving forever. Yeah, forever. Well, I mean, I wouldn’t say forever. We might, you know, if, for anyone who’s seen me, they might wonder about me moving to a sunny climate.

[00:01:41] Matt: Yeah. I thought you’d have been going further north. Really? I just, I get south. 

[00:01:46] Dara: Yeah, I get confused between north and south. I thought Spain was really cold and I’ve committed to it now, so. We’ll, we’ll give it a go. But yeah, moving to a small town called reo. I’m not sure if you No, that’s, that’s a, so you want 

[00:02:02] Matt: South Spain, which is Yes. About as hot as it gets. 

[00:02:05] Dara: It gets pretty hot, I hear. Yeah. So that’s, so anyway, that’s my news. Are you moving anywhere? Exciting. 

[00:02:12] Matt: I exist in a state of purgatory and AI panic. Don’t we all? Mostly flop sweat and AI panic is my weakness. 

[00:02:23] Dara: Yeah, we have, yeah, I’ve been, I’d be busy Doom mongering. This is my new favorite pastime.

[00:02:30] Dara: Well, look, I’ll, I’ll go with, I’ll go with pastime and just be telling people scary things. I, in fact, I don’t know if this podcast has parental guidance or I probably should because I seem to be telling people that robots are going to come and kill us. In our sleep. So we probably need some kind of, you know, warning, a spoiler for future episodes. Yeah.

[00:02:46] Dara: Spoiler slash warning on the podcast from now on. but yeah, there is a lot of, you know, AI existential dread or, or opportunity. 

[00:02:57] Matt: Is that not, is that not the absolute perfect segue into the salty news? 

[00:03:03] Dara: I think, I think, well, it depends. ’cause I mean, this week’s news might be all about Google Analytics.

[00:03:08] Matt: It’s very, pretty much all air. Is it ai. 

[00:03:11] Dara: Wow. Wow. Interesting. There’s a lot of AI stuff going, isn’t there? 

[00:03:15] Matt: I did, I went and looked at, I did, I actually, well, one of the things that is in here, which I’ll talk about, I’ll, I’ll kick this one off then as the first news item. 

[00:03:23] Dara: Have they changed the font color in Google Analytics?

[00:03:26] Matt: No. Well, exactly, but I, yeah, I used, I think last episode, if I get him episodes. Correct. Not to spoil the magic and theater of all of this, but we do, sometimes we call the intros out of order. But I think last episode we were talking about agents, the Agent tool that chats, GPT. Right. 

[00:03:45] Dara: And that’s it, we, it looked interesting that we didn’t have any access to it, which is the theme of this podcast. We talk about things we can’t actually use. 

[00:03:54] Matt: But this. But then shortly after that, we did get access to it. So I have played with it. And one of the things I did with it was said, go and look at Measure Pod, do a little research about what that is and, and the themes of it. And then off the back of that, go and find me a lot of news items that the host should cover in the podcast, based on what it is.

[00:04:15] Matt: And it pulled out a load that we already covered, which I think we just got from the transcripts. And then all of the Google analytics forms were very dry. It was very, it was very. The most interesting one was the Reddit ads thing that I think we’ve already talked about, but the rest of it was very, very dry. And it’s not, it’s not as interesting to talk about, I don’t think. I mean, it is in the day-to-day, but 

[00:04:41] Dara: it’s, and and, and look, it’s quality not quantity, isn’t it? There will be, we’ll, we’ll talk about the significant updates when they happen. We’re not going to, we’re not going to include it for the sake of, oh, let’s make sure we’ve got x number of this type of news. We’ll, we’ll just, we’ll go where our investigative journalistic noses take us 

[00:04:58] Matt: always. I mean, that’s, you know, why we are here. We’re, we’re, we are two professional journalists to just have that instinct to follow our noses. Yeah. 

[00:05:06] Dara: We’re, we’re going to win, is it the Pulitzer Prize? Nobel Prize? I’m not really sure what you get for journalism.

[00:05:13] Dara: I, I’m sure there’s going to be some prestigious prize coming our way soon for our, our hard-edged, what’s the phrase? I can’t even do phrases. Don’t pull your punches. No, no. Pull punches. Journalism. 

[00:05:25] Matt: Yeah. Yeah. Anyway, I played with it. It was interesting. We had a look at it last week as well as a company.

[00:05:31] Matt: Mark, co-founder went to order a pizza with it. It worked really well, didn’t it? You see? It did work, but it was just, it was just very slow. But my point was that you wouldn’t, you wouldn’t use it to order a pizza then just stare at it. Order a pizza. I imagine it’s meant to. Yeah. It’s meant to.

[00:05:50] Matt: Like, you go and say, get me a pizza and then go off and, I dunno, touch grass or ease or whatever. 

[00:05:56] Dara: Or eat a pizza while you wait for your pizza to be delivered. 

[00:05:58] Matt: Yeah. It’s tough to get the hunger for all that time it takes. So it did do it. It was interesting to see it do it. I’m trying to, I think we said last time as well, like, I’m trying to find the killer, the killer used for it. Yeah. At the minute. 

[00:06:11] Dara: I just love the, like, because it gives you the, and I guess it, it does want to show you the steps it’s going through, and you’re right. You wouldn’t necessarily have to sit there and watch it, but I just kind of think it’s, it makes me kind of smile to myself that it’s like the whole point is to take away tedious tasks and then it’s suggesting that you would sit there and watch it do the tedious task on your behalf as if it somehow then becomes entertainment where it’s like, I don’t want to order the pizza myself, but I’ll happily sit and watch a computer do it for me.

[00:06:37] Matt: I mean, that is kind of true the first time you do it because you’re like, oh, it’s in our whole tea. Yeah, we did. Well this was, it’s navigating a 

[00:06:41] Dara: website. This was the whole company watching Mark order a pizza. So I guess, yeah, I guess. Damn. 

[00:06:48] Matt: Actually it did suck us in. It was really good. I might, I might go back and watch it again just to get that thrill again.

[00:06:54] Dara: Wait till, wait till the weekend. 

[00:06:55] Matt: It was cool. It is cool. I’m going to keep playing with it and figure out what it’s, what to use it for. And then speaking of chat, GPT. PT five is imminent. It is coming. What do we know? It’s been, people have seen it like out in the wild, like hidden in bushes and things like that, but I, I swear that was seen, 

[00:07:19] Dara: I swear that was Chachi PT five I just saw over there.

[00:07:22] Matt: What was it, a Yeti, some sort? no, it’s, see, so people have seen it in, I think the cursor team, the people who make the cursor, IDE have like used it. Some people saw it. I think it accidentally got released into like a couple of beta versions of chat. GPT, Mark Altman’s been out there talking about the fact that it’s, that it’s coming in summer and then I think it’s been confirmed to be narrowed down to August.

[00:07:50] Matt: A lot of people who claim to have seen it have been singing its praises, seeing how good it is and that it kinda one shot pretty much any task you put in front of it. But we’ll have to wait and see. What it looks like. You, you’d think it’d have to be, they’d, they’ve really dragged their feet with putting a, a, a, a five on anything.

[00:08:12] Matt: Do you know what I mean? Like they’ve gone just 4 4 0 0 3 and then 4.1 and 4.5 and they’ve done everything in their power apart from calling something g PT five. So you’d hope it’s no pressure. This is a leap. Yeah. Yeah. Maybe they shot themselves in the foot and it’s just going to be actually 4.6, but yeah.

[00:08:35] Matt: Is it, people have been crying out for them many times. Just make it simpler. Timescales. Do we know August is all I know? Yeah. That’s all I know. That’s what my sources are telling me. Yeah. Are you asking for a GPT chat? Yeah. Yeah. You come out? 

[00:08:52] Dara: Yeah. Yeah. When, when’s your, when’s, when’s the new, when’s the new better version of you coming out?

[00:08:57] Matt: I. Yeah. By sources, I mean random LinkedIn posts and panic Googling, is normally my journalistic sources. It works, it’s always worked so far. So yeah, that will be very interesting when it comes out. I’m not sure. It’ll probably come out in stages into pro users first and with rate limits and all that sort of stuff, but we’ll see.

[00:09:20] Matt: And then the other thing that’s come out to switch over to Google, well, two Google, two Google News items. I found Notebook ln, which I think it’s fair to say we’re big, we’re, we’re big fans of, at Measure Lab, we use it quite a lot. It’s a way of sort of central chucking, a load of documentation of various different formats, like a website, a PDF document, a word document, a YouTube video, whatever you want, chucking it all into an interface.

[00:09:47] Matt: And then having like a, you can sort of talk and around that information you could even, you’ve been able to generate like a podcast off the back of it. So one. One experiment I did early on was to get hold of a GTM, JS om to download, which is just an A-A-G-T-M container that downloaded the JSO export and I just uploaded that in and got it to make a podcast around it.

[00:10:09] Matt: And it did and it was quite interesting at times. So anyway, that’s all cool. And they’ve now released videos so it can output a video based on what’s, what information you’ve chucked into, chucked into that thing. And I, I’ve been generating a couple today based on all notebooks I had, and it’s pretty, pretty impressive.

[00:10:32] Matt: It kinda outputs like this, almost like this slide talk. So it had, it broke it down into various steps and then it had sections in the slides and it sort of generated images and it seemed to pull out other information outside of that notebook, LM to contextualize things. And I’m pretty sure at one point.

[00:10:51] Matt: There’s an image in one of the documents that have been uploaded, and it had that as like an image in one of the slides that it was pointing to and a narration over the whole top, top of the whole thing. It sounded very natural. So it’s very cool. So if anybody, anybody does have access like me and other people apart from Dara, it’s definitely worth a play with.

[00:11:10] Dara: I’d love to see it. I just wish someone, I just wish I could, someone would share. I just wish I could see an example of it even. Yeah, yeah. No, it does sound really cool. and we are big fans of em. I, I, I kind of laughed a little bit when you said about the podcast thing. ’cause like I used to think you can’t fake enthusiasm, but those podcasts generate AI generated voices.

[00:11:31] Dara: You could create a podcast based on anything, and they sound so enthusiastic. It’s like, on today’s episode, we’re talking about GTM container 1 2 3 dash z Oh, it’s going to be a killer episode. 

[00:11:44] Matt: Yeah, yeah. It does get you engaged. They have it 10 to 11, but you can spot it a mile off. They have this very particular way of interrupting each other and Yeah.

[00:11:55] Matt: But it’s, it’s very impressive. We get, you’re so used to, to stuff so quickly that we are probably ai, we are, I don’t know. 

[00:12:00] Dara: Yeah. We’re generated, we’re just early. Early. Well, I’m a very early model. There’s been improvements since I’m a rogue. I’ve got, got released into the system. 

[00:12:09] Matt: Yeah. You’ve still got that rambley. You haven’t had the ramble, haven’t you? 

[00:12:12] Dara: I haven’t quite tweaked that yet. 

[00:12:15] Matt: And what else did I have? this is a, well, it’s all relevant, right? But Google launches AI search in the UK. Now, I don’t seem to have access to this, but apparently it has been released in the uk. I dunno if you’ve got access to this one.

[00:12:30] Dara: Nope. No. I should have lied and said I do, but no, I don’t. Yeah, you just instantly buckle us again. Follow the question again. I just can’t, can’t handle it. 

[00:12:40] Matt: This is. I think we’ve talked about this before, but essentially it is this long form Google answers to, to standard Google questions you’d go to. And this is in Google, it’s not like going to Gemini.

[00:12:52] Matt: You go to Google, you type in your question and it outputs a more long form piece of, long form answers with citations. And then I think of other results below it. and yeah, there’s a lot of controversy around it because, well, arguably it’s sending less and less people to actual sites and, and just give, presenting them with the information, which, if your whole business is built around content creation and selling ads and, and I dunno, affiliate links and things like that, based off of that content, it might be a bit of a death nail.

[00:13:30] Dara: Yeah. And I mean, in a, in a way, Google’s business is, or, or it’s built on companies like that. Then advertising on Google. Network. So it’s, it’s, it’s going to be really interesting to see, and I read in the, I think that, you know, the BBC article that went around, somebody, a spokesperson from Google was saying that they haven’t yet finalized how they’re going to monetize it or how, how the advertising on that is going to work.

[00:13:56] Dara: And it’s going to be really interesting to see how it does end up working, because, it’s one thing the, you know, if the organic web traffic reduces, but if there’s less ways for people to advertise against those results, then there’s going to be a lot of very unhappy advertisers and, you know, unless we’re missing something, Google is still very reliant on that advertising revenues.

[00:14:17] Matt: So yeah. A of very unhappy Google shareholders as well, probably. Well, yeah. Figure that out. Exactly. 

[00:14:22] Dara: Yeah. They’re walking a, they’re walking a tight rope, aren’t they? But, you know, they’re smart people. I’m sure they’ve got a plan. 

[00:14:27] Matt: Well, yeah, that’s the thing. Isn’t, that’s what, when you, when we’re talking about Apple, apple. Being laggards on? Or is it a cleverly disguised strategy? Like Google is really betting everything on this. It feels like they’ve, they’ve really put all their eggs in this basket. 

[00:14:43] Dara: Well, they all aren’t they? It’s like we talked about Oh yeah. Measure as well. And it’s like, and, and it, it could, could be one or two things, couldn’t it?

[00:14:50] Dara: It could be like, you know, the emperor’s new clothes. Everyone’s getting caught up in the excitement of it and throwing everything at it. And, it is a gamble, isn’t it? You know, these are big established businesses and if they’re going to completely turn overnight to be fully in on ai, it better work for them.

[00:15:10] Matt: And then the last piece of news I had, I’ve already talked about the fact that you actually got hold of chat, GPT agent mode now, but chat, GPT also released another mode called study mode. Yeah. I think it’s designed to, you can give it a subject and it can help sort of plan around your. Plan around how to teach it to you, so it’s less there to sort of give you the immediate answer and more to sort of construct ways of getting you to come to the answers yourselves. So I asked it earlier how to, how to teach me how to be a podcast host. and it said denied. 

[00:15:46] Dara: Sorry, this is beyond me, well. 

[00:15:48] Matt: You can already see the fruits of its labor. I’m a completely different animal. 

[00:15:51] Dara: This is beyond my computing power. You need to upgrade to the highest level of plan. Yeah. For this level, I had to pay three pounds.

[00:16:00] Matt: Yeah. But it, like, it ask me, what are you trying to achieve and what are your, what kind of podcasts are you hoping to run and create and all this kind of stuff. And he was asking me, what are some of your favorite things and what’s kind of episodes you want to do? He kind of just was just asking me a series of questions presuming the background, it was creating some sort of.

[00:16:24] Matt: Plan, attack and curriculum. I didn’t get that far ’cause I had to go and record a podcast, ironically. 

[00:16:29] Dara: Okay. So that’s the news for today. We’ve got a guest on today’s episode, which we’re going to go over now. so we’re joined by Roman Petro Chenko, who’s a lead marketing analyst at Car. Wow. Have a really good chat with Roman.

[00:16:43] Dara: It’s quiet, it’s, it’s wide ranging. but it’s, it’s focusing around, his client side perspective of implementing first party data strategy and some of the challenges that come with that. You know, we talk about the usual things you’d expect, like third party cookies and attribution issues and, we cover enhanced conversions for leads and capi.

[00:17:07] Dara: and then Matthew’s favorite subject, of course, we talk about the impact of ai, and get some Roman’s thoughts on where he thinks everything is heading. 

[00:17:15] Matt: Yeah, it was a great conversation, wide ranging, and I think. Definitely somebody that we could bring back to talk about a couple of other interesting subjects down the line.

[00:17:23] Matt: So enjoy. 

[00:17:27] Dara: So a very warm welcome to today’s guest, who is Roman Petro Chenko lead marketing analyst at Car. Wow. So, firstly, Roman, welcome to the Measure Pod and, and thank you for agreeing to join us. 

[00:17:40] Roman: Thank you very much for having me here. 

[00:17:43] Dara: So we always kick off, we let our guests introduce themselves so they can do a good job. So if you want to tell our listeners, as much or as little as you like about yourself, a little bit of background may lead up to what you’re doing in your current role at AT Car. Wow. 

[00:17:59] Roman: Sure. A hundred percent. It’s always the hardest part for me, honestly, because of how to summarize everything. But, I started my career in marketing about 12 years ago now.

[00:18:08] Roman: Started doing Google ads campaigns, Facebook ads and all of that, and agencies, you know, when Google Ads didn’t do everything for you back in the days without p max. Then both five years in, I tried a lot of different roles in analytics. That was back in Russia. I decided to move to Germany and this is where I started to focus on automation.

[00:18:27] Roman: And, an amazing world was open to me because I learned what BigQuery is and how amazing that thing is. And this is where I started to do a lot of Python swing around. And I realized that there, there is a whole role for that. It’s called marketing analysis. And I moved towards that role, worked there for a while, and about four years ago I moved to London and worked with Carroll.

[00:18:48] Roman: Now as I lead marketing analyst, I focus on two major things. One is measurement, paid marketing, everything that is about tracking, sending right events, making sure everything works well in bidding, but also everything on the backend for DBT attribution and how they disclose to them, to stakeholders. 

[00:19:09] Dara: Okay, so you’ve got quite a, quite a wide role, quite a wide remit there. A lot of things to cover. 

[00:19:15] Roman: But it’s very fun and I mean, you learn a lot on the job because, when I started doing everything like 12 years ago, most of the tools didn’t exist even five years ago, half of the tools didn’t exist. but it’s really, really fun. And I also have a very technical background, so it’s a bit easier for me to connect because I both did marketing campaigns and I can code.

[00:19:37] Roman: It’s very easy for me to talk to different stakeholders and be a translator or very complex things. 

[00:19:42] Dara: Well, we’re probably going to, and, and, and, and I like when we do this, we’re probably going to go slightly off on tangents in this conversation, which I think will be great. But there isn’t a kind of a, a, a kind of a main theme that we were going to talk to you about, which is kinda your client side perspective on first party data strategies.

[00:20:01] Dara: we, it’s, it’s, it’s somewhat rare that we have somebody, from the client side. A lot of our guests tend to be either from some kind of tech platform or, or maybe, people from. Agencies or consultancies. So we’re really keen to get your kind of insider view on how these things actually work, internally on, on the client side.

[00:20:21] Dara: So, but we, we will happily, go off on little detours, because I think we could probably talk to you about lots of different things. but if we kind of at least start around the first party data strategy and get some of your thoughts around how that actually gets implemented in practice versus maybe some of the, the theory that’s out there online.

[00:20:42] Roman: I was thinking about it like it’s, it all starts with cookies, right? As some people say, they should be called biscuits, right? it comes down to realization that third party cookies were supposed to leave us at some point last year. Then they didn’t, then they did again, then they got postponed again.

[00:20:59] Roman: But there is definitely a very clear sentiment that third party cookies at some point are not going to be as available as they are right now. So those that don’t know third party cookies is a fundamental tracking thing that allows different websites to profile users, understands what they do and everything around tracking essentially.

[00:21:18] Roman: And the amazing functionality. Like I was coming, I was thinking about the great examples of understanding the difference between the first party cookie, third cookie, for those that don’t know. And I thought, well, you come to Amazon, you log in, right? That’s Amazon setting something on your browser in the first party context.

[00:21:35] Roman: So it’s Amazon doing something for Amazon on the Amazon website. Good. Nobody really minds that. But then if you come to the car while and want to watch a video, on YouTube that is embedded on the page, that video is going to be loaded from YouTube domain on the car page, and then it’s going to appear in your history.

[00:21:53] Roman: So that appears in the history happening, because YouTube can read something happening on the car and that is a third party context. Though this particular example is very useful because you do want to remember what you watched on different websites. Also that comes to all advertising, everything advertising around makes it so that they can profile what you read, what you do, where you go and, well that’s not very safe.

[00:22:17] Roman: I mean, this particular technology is what, 20, 30, 40 years old. So at some point the privacy and understanding it’s very invasive as people say, is not necessarily great. To my knowledge. We already have Firefox and Safari Disallowing third party cookies by default. Right. Which reduces tracking by a lot.

[00:22:37] Roman: But, I checked just this morning, Chrome is still, 67% of it was about 70% of all browsers on the internet, and that did not turn it off yet. However, yes, that is about to come. Have you heard about the Google Privacy Sandbox, by the way? 

[00:22:56] Matt: Yeah, well we, we recently, they. It’s pretty hard to tell with Google, but they’ve, like you said, they postponed it a couple of times.

[00:23:03] Matt: Last I heard, the last postponement was indefinite. but we’re still working on Privacy Sandbox, but we’re just, they kick the can very far down the road. in, in, yeah, apparently. But yeah, so Privacy Sandbox looks interesting. 

[00:23:22] Roman: Yeah, exactly. So again, privacy Sandbox is, they’re a very fancy name for like, it’s, it’s, it’s, it’s a very complex name that I find that, that basically explains how the cookies are going to work out.

[00:23:33] Roman: But they have two different places where, describe it. There’s a website that’s called Google Privacy Sandbox. There is a Google help. They slightly deviate in what they tell, but essentially the whole idea, originally when they showed to us, not originally, but latest idea that I’ve seen about a year ago, that when you come to the website and that one of use third party cookies is going to pop up the, another popup in front of you.

[00:23:55] Roman: But then if you can’t do it for every single pixel, for example, it’s going to have a separate popup of Facebook, separate popup for Amazon and everything like this. it’s very hard to say whether it’s going to happen or not and when it’s going to happen because again, like it was supposed to happen but didn’t.

[00:24:10] Roman: But I think the whole quality of tracking purely on cookie base is still going down a lot because we already lost a huge, iOS and safari world. Right? 

[00:24:20] Matt: Yeah. I mean, that’s the problem, isn’t it, that half of, from a mobile, like 70% of browsers, desktop browsers are Chrome. But you’d imagine that iOS has a much better, bigger chunk of it with Safari, and they’re limiting cookie lifetimes and doing all sorts of stuff to keep their privacy war going. Is it a war’s probably the wrong word, but the, the, their fight against. Yeah. 

[00:24:46] Roman: Another very big part of there is like an app browser. So you can have a safari, but then you can have Gmail open the link that’s going to open a Gmail context. And that wouldn’t have particularly any storage or retention essentially from purely coming to the point why it’s important.

[00:25:04] Roman: It’s, it’s, it’s, it’s somewhat of a nightmare nowadays to have cookies purely, purely third party cookies to track users. So that’s why, first party data comes into play with, well, it’s a very simple idea, right? we barely change our emails in our lifetimes. They all stay roughly the same. I have the same email I had 15 years ago, and if you can securely share that across the browsers, sorry, across the platforms with the consent, that becomes significantly easier to understand what people do and optimize campaigns.

[00:25:37] Dara: So how do you then, okay, so, so we’ve established why. It’s important. So what do you, if you, if you’re still, if you’re a business, and so this is probably going back a little bit in time for you, but if you are, if you’re a business who’s still maybe at least somewhat reliant on third party cookies and you realize, okay, we really need to care about first party, what’s the kind of process you’d go through?

[00:26:00] Dara: How do you, what are the steps you’d go through to kind of move your reliance away from, because you can’t, you know, it’s not an overnight thing, is it? 

[00:26:07] Roman: It’s anything but an overnight thing. I would probably say, the, the biggest thing there, that, that, that starts even before marketing starts is that you need to position your product and your website the way that it helps you to collect the first flight of data in the first place.

[00:26:22] Roman: Right? If you don’t have any way for people to sign up, if you don’t have any way for people to stay retained with your business, then there is no reason to share emails and it, it sounds very obvious, but on the client side, what happens a lot is. When product, when you try to optimize product and the conversion flow, you try to maximize user flow and, and remove any barriers.

[00:26:42] Roman: And in a lot of cases, the signup flows become barriers. So it kind of comes in exchange of what you want and every business needs to understand at which point it becomes very important to retain the customer base. Obviously it comes with a lot of benefits, like CRM management, you can come with, CTP platform, like there’s a lot of benefits of using it.

[00:27:06] Roman: But the first real challenge comes at the point that does your product allow it? And if it doesn’t, what should be the steps that you, you should be with your product to start collecting the first party data. but once, imagine, imagine we have that, right? Imagine we have a product that collects that.

[00:27:23] Roman: The second big piece is you already probably have third party cookies. You already have measurement. What I would not recommend is going there, like, okay, let’s just change it overnight and change all the bids, you know, every platform. Let campaigns go. Pan Bananas, budget, go off the p and l because that’s, that’s where it really, the client side is you have usually the flow and campaign and budgeting set up way upfront.

[00:27:47] Roman: And because you’re changing a fundamental tracking thing, it’s going to change how campaigns are optimized and how tracking works. You need to be ideally doing it in stages. 

[00:27:58] Matt: How, how far down that particular rabbit hole are you, at Carlile then? Are you, are you at the other end looking back on your glorious achievements or are you in the midst of, of those, those changes?

[00:28:14] Roman: I think we are very, very well established. We have done quite a lot of testing. We’re using first party dates, definitely everywhere we can use it. But I think where the real difficulty comes in is, the first flight of data sounds like a box. You collect it and you send it, but then you come to all the details and you realize that every platform has its own specific use cases and specific things you need to collect.

[00:28:43] Roman: One of them that specifically threw me away is if you look at what Google ads ask you to send in as, as a first flight of data through the API uploaded, it asks you to change Gmail to google mail.com. And if you miss that line, you kind of throw a huge punch of emails and it’s not going to be recognized. I’m not sure if it’s still there, but half a year ago it was there in hell. 

[00:29:07] Dara: Because it used to be Google Mail, right? Because I think when I first signed up it was I had a Google mail and then they, at some point, switched to gma. So this must be something real like an archaic buried system where it’s code somewhere.

[00:29:19] Dara: Yeah, it must be Google Mail. You know, the original, the original format, not this new shortened version. It’s not Hotmail. No, no it’s not. so, just going, I dunno if this is a back, back, a step or not, but if you, so you mentioned earlier about like, you gave the example of Amazon and you said, you know, if you go on Amazon, it’s in your interest to be signed in and you’re happy about that because you’re getting kind of recommendations and you’re getting to see what you purchased before.

[00:29:48] Dara: But a lot of businesses don’t have that luxury and, and maybe there’s a point where they capture that email address where if somebody actually successfully purchases something or decides to sign up to a mailing list or whatever. But is there anything you can do around what’s happening before that? So if, if somebody’s, if you’ve got the type of website where, I mean this might even be the case for cars Wow.

[00:30:11] Dara: Where you’re going to have a lot of people, you’re going to have a lot of tire kickers. So how do you, is there anything you can do to try and encourage those users to part with an email address or, or, or some other, identifiable piece of information? Or do you just have to accept that you are only going to get that identifier once somebody’s far enough through the funnel?

[00:30:34] Roman: I think there are a lot of different layers of that question because, you can get, collect a lot of emails, but there are also specific regulations that you have to follow. And there are certain consent regulations and reasons how you can process and, and send it. And also in a lot of cases, you’re not necessarily want to.

[00:30:54] Roman: Creates so many barriers to just collect, like first party data is absolutely like gold. Like you want to collect you, you can’t, you want to identify as many users as possible. But if the user, let’s say, parses the email, but they don’t necessarily become a regular user of your product, they just log, for example, they sign up for something, but they leave and they do not log in.

[00:31:17] Roman: And then tomorrow they come from a different browser or anything that doesn’t necessarily help you in any way to identify that user. And this is where I start with like, your product should create an incentive for people to sign up for them to actually get some benefit. So for a car, for example, when you think about buying a car, you really, like, if you buy a used car, you don’t have a luxury to pick specific color tires, place like, price, all of that.

[00:31:43] Roman: You, you monitor the same way that if you buy something on Right Move or Zoopla, you, you, you, you sit there, you monitor what was available, you find them the right thing. We have a lot of different flows that help people to make better choices. but like, again, not every, like, some business is going to have it a bit harder away.

[00:32:02] Roman: Some businesses are going to have it a bit easier depending on what the business does. But that, that reason for people to come back, that’s very important. 

[00:32:10] Matt: I think it’s time for the first detour into ai, so I’m going to do that. Wow. Necessarily, I thought it was a bit late. It’s normally within the first couple of minutes, but it is related because we’re talking about the, you know, in relation to tire kickers or people coming and perusing these sites and, and putting in front of people those opportunities and to, to collect that first party data.

[00:32:34] Matt: What have you seen like, being at a company like Carlile that functions in that way and is that kind of catalog searching, maybe long tail way of, of purchasing. How has AI affected or begun to affect that? Because I can imagine, you know, some of the new things that are coming out with say, ChatGPT and stuff where they’re beginning to surface results for car purchases or, or handbags or whatever it may be in the actual chat interface themselves.

[00:33:04] Matt: I can imagine some of that being abstracted away and, and you lose the opportunity to collect first party data ’cause they’re nicking it, over that side. I don’t know if you’ve, if, if, if you’ve seen any trend yet or if it’s still early days. 

[00:33:17] Roman: I mean, it’s not even Carroll specific. I think this is, there’s quite a lot of conversation in the UK specifically right now and its AI overview from Google.

[00:33:25] Roman: This is where a huge kind of conversation happens. I have been to very nice calls, meetups with CloudFlare and CloudFlare provides statistics on how much time a, a system, AI system crawls your website versus how much time SNS senses you as a person. How much is the difference between one and another?

[00:33:47] Roman: And I don’t necessarily remember the exact numbers, so I don’t want to quote them incorrectly, but if you look at the chart, GPT and like their, the 10 of AI agents, they are in a very, very red area in terms of they be like a thousand requests before the user actually comes to your website. But they’re very small.

[00:34:05] Roman: So like, it is small in the context of most people who don’t use JGPT normally for day-to-day things. They still open Siri, they still open Google and stuff. However, they, Google AI will review. That’s where you, like a lot of companies, especially publishing companies that basically earn money on people coming and seeing ads start realizing that people don’t necessarily follow after the I review, especially when it comes to search terms, where like they’re more informative in research.

[00:34:35] Roman: Like, is that car good? Is this good? When it comes to cargo specifically, because you, you like, it’s, it’s closer to the point where you also want to see the cars, compare the prices and all of that. It’s a little bit easier in terms, but we definitely see how AI overview starts to take these informative queries and serve that upfront.

[00:34:56] Dara: So, drilling back into the, or kind of key to go a little bit into the detail and you said about the, the kind of technical, process involved in, moving to more first party data approach. So what’s the, I know it’s going to differ depending on, you know, what marketing you’re running, what platforms you’re using, but can you kind of give us an overview of what the kind of steps are and what the, maybe if we start with like, what’s the kind of minimal, minimum kind of viable tech stack that you can use to have a, a good first party data strategy?

[00:35:28] Roman: I’m going to start with one thing though. I have to tell them. Yeah. I’m not a legal advisor. So when you talk with work, the first vital data, please consider that you need to talk to a person who can give you legal advice. and there are a lot of regulations depending on the country. Like in the UK we have ICO, we have GDPR.

[00:35:45] Roman: In Europe, there are different regulators as well. So leave it, be considering that. but essentially coming to what steps need to be done, the simplest step and simplest, because most people are going to have online tracking. So they’re going to have some GTM set up, they’re going to have some conversion actions coming.

[00:36:01] Roman: There is to enable, essentially enhanced conversions. They used to be you sending an email, hashed email and some other data when the person converts at that point. And now they migrated to a different solution, which is, I think they call it an open conversion event. So essentially when the person logs in and gives up their email, considering there is a consent given, Google sends that information, Google Pixel sends that information to Google immediately.

[00:36:29] Roman: That becomes a context of everything else the user is doing on the website. So you don’t necessarily send that with every conversion. You just set up one thing. It’s very easy to set up. All you need to do is to go to GTM, I hope you have a GTM or any similar tag solution. But, it’s called a user provided Data Event or something like this.

[00:36:50] Roman: There are not so many of them. It is very easy to find inside GTM. And then all you need to do is to expose the email, hash email in a data layer. This is where it’s probably like, oh, all I need to do is this, but how do I do that? This is where you probably need some product or engineering support to help you do that. Unless you’re using some built in, built-in systems like I haven’t worked with Shopify, but I would imagine those systems have it built-in. 

[00:37:13] Matt: Just to check out a word of caution there, because we’ve actually had a couple of clients who use the user provided data function within GTM and they’ve come to us with all sorts of problems like massive drops in users and then single users purchasing.

[00:37:30] Matt: Hundreds of thousands of, of, of products and, and lots of, lots of revenue attributed to individual users. And what it turns out was that they’d set up, in a couple of cases, they’d set up, user provided data and it was grabbing a random other email address off the page, hashing that email address, sending that off to Google.

[00:37:54] Matt: And then Google was seeing that as an individual user pulling all of those different transactions together. And they’re just making these Uber users within, within, within GA fours. I, I think, I mean I think it’s in beta anyway right now, that function. But it’s just something to be careful of, like really carefully testing it to make sure that it’s, it’s, it’s behaving in your, and, and the data looks correct when you do it. ‘Cause it can be a, it can be tricky to unpick and figure out if it, if it does go a bit awry. 

[00:38:21] Roman: I would imagine this happened if you do automatic detection right. Of emails. Yeah. We don’t do automatic detection, but it’s again, like it’ll depend on your company. Do you have an engineer? And this is where a lot of, like, from the client side, literally a huge part of it is making sure you talk to people, you build a case internally, you get buy-ins.

[00:38:40] Roman: It’s probably half of the work that you need to do to get something adopted. But yeah, if you’re, if you’re a smaller company, you’ll have to be careful because you probably will opt in for something easier, but wait, wait quicker. 

[00:38:52] Matt: Yeah. I, I wonder, did, did at any point you, I mean we’ve, we’ve seen a couple of larger clients now, and larger companies that have tried to pretty much remove any middle step from that first party data.

[00:39:08] Matt: So it, it, the, the sort of what you’re describing there, you’re still sending that data to a third party before then say, routing it into BigQuery, where you kind of own it in a first party context. It still goes to a third party for an analytics platform and reporting and stuff before it’s then extracted.

[00:39:24] Matt: Have you, have you, have you had any consideration of just cutting out the Google analytics piece and just sending the data straight through to, to a cloud data warehouse to completely own that first party data end to end? 

[00:39:37] Roman: We do that. We do both, yes. So we, because we have a production, data coming, so essentially not everything coming when you come, when you try to buy or sell a car, not everything happens in the browser.

[00:39:50] Roman: So a lot of this is coming, from the backend as well. The simplest way to think about it is, if you sell your car online, the actual sale is not going to happen in your browser while you’re sitting there in front of your computer for two and a half days. Or for simpler businesses, it’s going to be a qualifying lead.

[00:40:06] Roman: So you can get a lot of leads, but only some of them are really good. And if you want to send that data back to Google so you would use it, you use OCI, which is a fine conversion import, which would require you to have a G lead. Now you would probably use a C four lead, which is essentially, you can send either email or click id or both of them, which now also accepts so many different click ideas that I can’t catch up with them, but yeah.

[00:40:31] Dara: so, Matthew obviously gave a few examples of things we’ve experienced where some of our clients have maybe got things a little bit wrong. But the way you explained it, and I know you for the sake of the time we have, you kind of simplified it a little bit, but you made it almost sound quite easy too.

[00:40:46] Dara: move to enhanced conversions. So did you experience any issues or is there any advice you would give people for anything to watch out for? Or, you know, did it have any knock on effects on your attribution, for example, that you had to kind of unpick? Like, what was your, what was your experience of actually implementing it and then, and then using the data that resulted?

[00:41:06] Roman: Very good question. With a few things like, please stop me when, when I go too much into detail, but I think depending on how big you are, like if you’re a big company, your biggest challenges are going to be to get buy-ins, like the legal approval and product approval. Because what you’re going to have probably is a website that has 50, like a hundred different pages.

[00:41:25] Roman: And the idea that the person is logged in, technically doesn’t exist as a context on every page. So if your page is serving information that is not user related in any way, for example, it’s a block or review a lot of websites not going to cache information about you being logged in because it’s relevant.

[00:41:42] Roman: so if you’d even wanted to track that first party context on that page, you not necessarily can do that. And then you become into this very big difficult conversation. Whereas if you’re a smaller client, your bigger complexity is going to be going to have smaller, way more understood website, but then building that functionality, making sure that you have GTM, that the data gets there, that legal understands what it is, and measuring that becomes way hard, like depending on where you are.

[00:42:09] Roman: the thing that I would definitely look after for, so for example, we used to use offline conversions and we switch to see for leads, which is essentially very similar concept, but you, you have additional data and you have more freedom in what you send. to do that, you need to implement this first party, user collective event online, right?

[00:42:31] Roman: the moment you implement that, it switches a hundred percent of all your client conversions to receive for leads because it doesn’t actually care where the data comes from as long as it can stitch it. You can’t do the ab test of conversions there, you can only do pre-post. And the only way we figured that out is because we, we kind of restarted our test multiple times.

[00:42:50] Roman: We had enough historical data and we saw that the moment we implemented that event, all of our conversion actions just collided in a single line collapse in a single line. So if you, if you wanted to measure that, be careful, you can’t necessarily measure, you can’t do one conversion in the C leads because the C leads require that online event.

[00:43:10] Roman: The moment you do an online event, all of them become C leads. That’s probably one of the very trickiest things that I did not expect. 

[00:43:17] Dara: And what about others, so we’re, we’re talking a lot about Google, but what about other, like things like CAPI and, and and, and the other equivalents. What’s your, what’s your been, because I hear a lot of, you know, it, it seems to, you mention that word to people and you can see there.

[00:43:32] Dara: You can see the effects of the, you know, almost like the PTSD when you talk to people about CAPI sometimes. So what, what was your experience? 

[00:43:39] Roman: We actually started using CAPI before we, well see, it only came out last year, right? So CAPI was way, way before that. you asked me in previous time about attribution.

[00:43:50] Roman: So we have internal attribution and then we have, we kind of like separate attribution and if it is a little bit from how it works we’re going to copy. The complexity comes in that copy works very differently to how Google Ads works in terms of how it shares the conversions and what it takes into account.

[00:44:08] Roman: One of the things that Copy allows to do is to take online event, offline event and merge them together and basically both use both of them to maximize the understanding of how campaign works, whereas you can’t do it on Google and we internally did not really know what to do with that. Like how do you measure them, like essentially it’s very hard to measure.

[00:44:28] Roman: Is it good, is it bad? Another very interesting thing is that Google, like Kathy, asks you to send all events within six hours. The sooner you can do it the better. And then do not send events longer than seven days after they have happened. Which for again, if you’re sending something qualified lead or cancellation of whatever, like anything that takes physical time to process becomes a big problem.

[00:44:51] Roman: And then you come to Google Ads and it asks, please do not send us anything within the first six hours. You’re not ready. That is a very, very interesting thing because like you need to look into that. And what we found out is you can’t have a universal solution. You have a tool like capi and you try to make it work the best in the context of Facebook.

[00:45:10] Roman: And then you put the measurement of Facebook, like understanding incrementality and and all of those things as a separate question to how the tool works, which again, becomes very, very difficult if you try to merge it all together. 

[00:45:24] Dara: So how, I mean, maybe I’m jumping ahead a little bit here, but how on earth do you? Manage that, the communication of that internally. ’cause I think if you, if you think back to like the simpler times where maybe people just used something like, like GA and they pulled all their, you know, Alaska attribution out of there and there were differences in terms of how campaigns were run, obviously in the different platforms.

[00:45:46] Dara: But, you know, the reporting might have been quite simple and now you’ve got so much nuance, and you’ve got not only all the complexity just around consent and the gaps that, that might leave in the data, like broadly speaking, but then you’ve specifically got all of these different ways that all the different solutions work.

[00:46:03] Dara: I mean, that must be, I can only imagine what that must be like for you trying to communicate that in a, in a way that could be understood by, by different types of stakeholders who will have different levels of knowledge about, about how, how all of this works. 

[00:46:19] Roman: It is a challenge, and it’s a challenge both ways because you’re trying to come there and say, Hey, I have an amazing tool.

[00:46:26] Roman: Or like this, this thing is amazing. But we have targets, we have everything. And if you ever like to switch the big campaign, Google ads to a new bidding strategy, it’s never smooth sailing. It’s like, it is going to work. Is it not going to work? What’s going to happen tomorrow? Like it’s, it’s, it’s a very, very scary thing.

[00:46:42] Roman: And, building the right communication and explaining to people why you do certain things is very important. The first time we switched to value-based bidding, it was a level of communication. Like you talk to the team, we combined together, like together with marketing efforts. Like we, we want to do that.

[00:46:59] Roman: Okay, who do we need to get on board, how do we do that? And the same happens with pretty much every iteration because again, like bidding, switching, bidding, especially when you have everything working in a way, it’s like you, you are trying to change a tire while you’re driving far. Like it’s very difficult.

[00:47:17] Roman: but it’s also difficult for us from perspective. You come there and like, okay, there is this amazing thing that came out, let’s say cap or you c if you see 40 is anything. How do you pray? How do you say I wanted to spend three, four weeks of work understanding it, being in your pipeline, building data tests to measure the impact of it where there is no real understanding whether it’s going to be of any benefit to your business.

[00:47:40] Roman: answer for that. Honestly, I don’t know. You just try to find time and build cases together. 

[00:47:47] Dara: And then I guess a follow on or a related question. So there’s, there’s internally communicating it, but then also in terms of testing how it works, it must add a lot of complexity when you’re trying to test.

[00:47:58] Dara: ’cause you, you’re not just testing the site or the, or the app, you’re, you’re testing campaigns with different, that are tracked and measured in different ways. And I mean, it is a very similar question, but in terms of how you actually run experimentation, like, is there a kinda way you’d navigate through all of these different complexities and nuances with the different platforms and.

[00:48:24] Dara: And even just in terms of measuring experiments in general, when you’ve got all of the, you know, potential gaps in the data around consent. 

[00:48:33] Roman: It’s, again, it’s, it’s, it all comes down to really, like, you have to know the platform really well. Like you need to understand what’s happening behind the hood to or under the hood and like what’s happening in there too, to build a good measurement plan.

[00:48:47] Roman: When we look at the Google ads, for example, with Google Ads, it’s somewhat straightforward in terms of how we set it up. We have a quite complex pipeline. I’m not going to like it, and I’m getting lost sometimes in it because we try to collect, connect both online and offline together, and then measure it. But then the benefit of it, you create a new conversion action and then you can compare existing conversion action to a new conversion action.

[00:49:10] Roman: And it, you don’t need to switch anything to it. You can see the breakdown campaign back and play, and then you need to build a story. So for example, pre C for leads, a lot of the things we have seen at the moment, you turn it on, it starts using first by first by the data. It shifts about like about 5% of our conversions between different campaigns and most of that shifts were like pm a got quite a nice shift and things like this.

[00:49:34] Roman: But when you completely utilize it fully so you not start, like we, in the first step, we just send click IDs and first by the data, but there is no click I, we all send first by the data and in the second one we’re like, okay, let’s just see what we have outta everybody who can send it. Let’s send as much as we can.

[00:49:50] Roman: And then when we saw, like when we did that, we saw that, for example, demand gen has grown by like 24% versus search, it has grown by 5%. So like, okay, this thing is really good for cross device teaching and probably like understanding impression based data and how people like to avoid the mid funnel interact with mid funnel campaigns.

[00:50:09] Roman: But the question is how do you feed that into your attribution? So how do you say like, this actually is really good at understanding what people do outside just clicking, but you will measure it based on the click based attribution. So things don’t connect with that thing. Honestly, right now with this question, I’m trying to understand how to, how to build a case and how to find the truth, but it’s quite difficult.

[00:50:32] Dara: Yeah. I’m really interested in that. If you get somewhere with that, we might have to get you back on to, to talk us through because it is, it’s such a big question, isn’t it? 

[00:50:40] Roman: It is and it’s the same as MM question, right? Everybody wants to do MAM but at the same time, running incrementally and making different budgets and all of that takes a lot of time and a lot of money.

[00:50:53] Roman: And it’s also like, it’s only going to show that your things are working as good as your creative design. You landing pages and everything else. If you run an ad that is not that good, it’s going to show a low income mentality and you’re like, oh, the channel doesn’t work. Well, the channel works. It’s just you need, you need more than one point of data to, to make a story.

[00:51:10] Roman: And I honestly think that’s everybody’s problem. It’s just very, very difficult depending on where your company is in this life cycle of data and measurement. You need to find what works really best and then try to build a story around it. 

[00:51:25] Matt: What has, has there been any sort of discernible change in who, who you’ve needed what, what your team, what teams look like?

[00:51:33] Matt: with this, with the shift over from sort of the old world of collecting data and building out with, with third party cookies to first party data? And I asked that question probably partly from a thinking about security and the additional need of needing to hash the data and make sure the data’s secure and all the extra governance headaches that might come with first party data.

[00:51:53] Matt: What kind of skills do you need and has that changed, do you think? 

[00:51:58] Roman: I can only sell what we have internally, like internally what we have. Obviously legal, I have mentioned it multiple times, so every time you work with it again, you need to make sure that everything is compliant. But now the next question comes in.

[00:52:10] Roman: As, the product. And look, we need somebody from the people developing the business or developers and engineers to understand why we need that and where it needs to be present. And now we develop new pages. They always need to understand, they have an understanding of it, like a very common thing.

[00:52:25] Roman: For example, you can come there from SEO pages and they’re going to make a permanent redirect. So like this page moved their page. So Google understands that. But this redirect can be implemented with, saving gcl and without saving a gcl. And if you do it without saving gli, you’re losing all the information about people coming through Google Ads because you’re also losing UTMs and everything else.

[00:52:46] Roman: That particular project, like finding that out, that exists in certain pages and fixing that requires quite a lot of investigation because you’re trying to find something that physically doesn’t show in your data because the data is already lost by the time you’re measuring it. Another side of it is because we use this qualified leads concept.

[00:53:04] Roman: If we have, we process data through DBT. So you need to know how to process the data, but also we have a reverse TL, which is essentially a very simple thing. It takes data from the warehouse, send it to Google ads. But to do that, somebody needs to read the API documentation. Like what do we need to send? Can we send that?

[00:53:21] Roman: What, what Tam? Like it’s a very simple set of questions you need to answer, but unless you really know that you’re probably going to go to ChatGPT and like, okay, there’s this 50 fields, I need to understand which one I actually need, which one I can use. Like this is where it becomes, I would probably say quite complicated. These nuances. 

[00:53:42] Matt: We had, Chad Sanderson on, on the podcast a couple of weeks ago and he was talking about data contracts, to try and make sure that everything across, like every complex bit and every different, Dependency from the data collection to right the way through to sending the data off to a third party platform.

[00:54:03] Matt: Everybody has ownership, understands what they’re doing and changing will downstream affect all of these other things. Do you have anything, any, do you have any sort of similar processes at, at Carl Wow. To kind of make sure that everybody is singing from the same hymn sheet and a developer isn’t going to change something on a page that fundamentally breaks your tracking and campaigns and everything downstream or, you know. 

[00:54:23] Roman: Like every rule is written essentially. Exactly. We do have a lot of ownership. it did not come as a front thing. Like we broke a few things here and there. We realized, okay. That cannot change. So the tech, like we have attribution internally that runs in sql, it’s, it’s quite a common thing for the companies of our style and size, that.

[00:54:47] Roman: Sequel, like we made a mistake once in it, and then every downstream dependency obviously went off the rails and now we have a distant framework for it. So every time we change, you can change anything anywhere else, but if you change anything in there, you have to follow this particular guidance step and make sure that, obviously it creates an extra weight you need to pull every time you make the changes.

[00:55:08] Roman: But it is just because you have a cost of improvement versus the cost of failure, you now need to evaluate that. Where the product was the same thing. There are certain things about the path of the product. So for example, if there is a funnel product that can change the funnel the way they want, but the final step of the funnel should provide this, this particular field and this particular information.

[00:55:28] Roman: And if they want to change anything around that, then we have an internal discussion. Like, okay, what’s the implication? I would probably say the most difficult thing for people, like in, in my role is to have that mindset of. You want a product to change things and you want people to make it better, you just need to always think about how to connect it together.

[00:55:49] Roman: Because at some point you might like to sidetrack into the thought, oh, I don’t want anybody to change anything because I just finished working on it. But that’s not true. You do want funnels to improve and sometimes it requires certain steps to be, go away as certain data to most, it just requires to constantly adapt to that.

[00:56:06] Matt: And how do you, how do you manage that? This is a bit more of a technical question, but in terms of that framework, SQL framework and making sure you’ve got versioning and are you using like A-A-D-B-T or a data form to, to sort of manage, well use DBT, use DBT and and what, what was your, did you, did you have a choice there to make between data form or DBT or, or was it just always DBT is what you went with?

[00:56:29] Matt: I’m just interested ’cause if you got data in BigQuery, what, what the, what the decision was there? 

[00:56:34] Roman: we don’t have a BigQuery, we have Snowflake. 

[00:56:35] Matt: Ah, okay. 

[00:56:36] Roman: Yes. So BigQuery is something I discovered for myself. I absolutely love the query. When we have Snowflake, I think DBT is a very mature product. And we, when we started migration, when I joined the company about four years ago, we had Python.

[00:56:50] Roman: So we still had version controls, but everything was running on Python. And with Python, you don’t have this lineage and data plus tests and everything. Migration is always difficult because you need to move a lot of things and you need to adapt to a new way of thinking. But it comes with a lot of benefits.

[00:57:08] Roman: Like me, I honestly think DBT is one of the best things that’s ever existed for my job. I love it. But it comes with a lot of very fun things. How do you orchestrate very complex things? So for example, my team looks at about 50, 60, 75 tables and they’re all cross dependent. And how do you make sure which ones are important, which ones are not important?

[00:57:28] Roman: So you start adapting some rules. but just to throw a little bit of less technical thing in it. When they just came in, we’re like, we’re going to test everything. We’re going to do this data engineering driven idea. We’re going to test everything, everything’s going to be really good. So we put a lot of tests and now our two schemas, like two main projects, have about 1,200 tests to run on top.

[00:57:49] Roman: And every test takes time to run on, in the warehouse. And all of them are not necessarily needed, but you pay for every test. So now we’re now at the point where we don’t want to test everything. It costs a lot of money. We need to think about what we actually want to test? And what are those things? 

[00:58:06] Matt: Yeah, that’s amazing. One, one 1,200 tests. That is thorough. But yes, I’m very computational. Yeah. 

[00:58:14] Dara: Yeah. Interesting. How do people Roman, with, with, with everyone, sorry, with everything that you’re doing and, and all it, like, it sounds like you’ve got such a great handle on all this, but you are obviously one person within the company and you’ve got a whole lot of stakeholders.

[00:58:28] Dara: So I asked earlier about kind of like how you communicate, but. Kind of flipping that a little bit, how, how are people consuming data? Like, how are your colleagues, are you, is everything getting kind of funneled into a set of dashboards that people just look at? Or is there, is there more of a, an, like, are you, do you go down more of a self-serve approach or like, how, how is data actually consumed outside of just your, your role?

[00:58:54] Roman: Just to clarify, like, I’m not the only one who built all of it, right, of course. Yeah. We’re looking for a huge part of it. But, it’s the team, my team and my colleagues are absolutely amazing in pushing it forward. and, and yes, we argue a lot about how to orchestrate certain things, but that’s where the cool outcome really comes.

[00:59:12] Roman: when it comes to self-serve, I think this is like the very literal problem of every company on the market right now. You have so much data. How do you make it available and how do you query, how do you query it? we use everything, like a little bit of everything. We have bi tool, we have.

[00:59:30] Roman: Predefined reports. We have specially trained G Sheets. G sheets are always the best thing in the world. but also what I found is that if you build a very specific table and you have a stakeholder that really loves what they do, so for example, if you think about Google Ads, a lot of people that do Google ads, they know a lot of technical things because you just have to learn all the settings and campaigns.

[00:59:53] Roman: And at some point they open Google ADSD scripts and like to start to dig into that. I found that it’s very easy to give some sql, a predefined sql you can run because like, just for the context, some of the tables that we have are hundreds of million rows and you can’t just go and send it to G Sheet. You can’t like it, but you just need some very narrow scope of data.

[01:00:13] Roman: So I need some internal training for some of the stakeholders to, to pull the data, some of the things we do for them because it’s a bit too difficult, but we all know where it’s going to come. Right. With AI and everything, I would imagine in a couple of years we’re going to have a very difficult conversation.

[01:00:31] Dara: Yeah, I think that’s where my mind was going for the next question. And, I’m curious, are people now using, like, are people using code assistant AI tools too, so you said you’ve given them a bit of a sequel. Are they, now, are people starting to think, do you know what, I don’t need to bother Roman with this.

[01:00:46] Dara: I can just get, you know, copilot or, or whatever to write some sequel for me. 

[01:00:51] Roman: I really hope it’s going to shut soon. I don’t want them to stop bothering me. I absolutely love working with ’em and I absolutely love helping people. But, you already can do a lot of things. You can go to charge GPT, you can go custom GPT, you can upload the, the schemas and SQL of your tables and say you are a co assistant, helps marketing people to do this and this and this.

[01:01:13] Roman: write SQL in this style, do this and this. And. I know that because they did the same thing for Google ads scripts just a week ago. So if you go there and try to do Google ads scripts straight away, because it doesn’t have enough knowledge, it starts to confuse Edwards app with ads app and just doesn’t have enough data.

[01:01:31] Roman: So putting some effort in, we have now a simple agent. It didn’t take that much time to do that. Can write really good scripts, like not very complex. I still need to know things, but I did four prompts and then four prompts. I got three scripts that are working from the first attempt and I think this is really good.

[01:01:50] Matt: Yeah, that’s what I found, I mean just just from a raw LLM perspective, I’ve found over the last sort of six, seven months that the SQL it produces has increased, and has got a lot better quality like it used to be. It just didn’t know BigQuery sql, for example. And it would, it would always just be chucking in random.

[01:02:12] Matt: functions and, and transformations that just didn’t exist, but now it seems to get in the context more without me even touching it. and then you’ve got MCP servers beginning to appear, like the Google released the Google Analytics one the other day and there’s 20 big query ones out there and GTM where you, I, I imagine there’s going to be context baked in with all of the various functions and tools and API calls, but then, you know, that’s dangerous.

[01:02:40] Matt: ’cause then you, you, you’re not just giving ’em the ability to query, you’re also given the ability to run other things as well, potentially. But it is, it is a scary and interesting future it feels like. 

[01:02:52] Roman: I mean, I think a huge part of the challenge of the job nowadays is to find the time to do things you need to do and not try to play around with GPT.

[01:02:59] Roman: Like I found the GA four and I started like, try it out and plug it in this week. And I’m like, this is amazing. Like half a year from now, you’ll just. Plug it into Slack and, and, and, and just, it’s just going to pull the data for you. It’s not very quite yet to do it fully, because now if you try, I don’t dunno if you tried it, I tried it and you ask it to do it, like, I messed up the, the, the API column, I need to redo it and it kinda like two, three times and eventually it gets it right.

[01:03:29] Roman: But it takes a lot of compute power. But I’m pretty sure that’s where we are planning. 

[01:03:33] Matt: Yeah, we found it, we found it useful for sort of context building, not necessarily going and taking actions right now, which it can do. Like you say, if you, if you’re specific enough, it can go and do it, but I found it useful if I’m just dropping into a, into a container or an analytics Google analytics instance I’ve never been in before and just like, what’s going on in here?

[01:03:54] Matt: I found it’s good at just giving me a bit of context that would’ve taken, you know, four or five hours of digging and figuring out within a few minutes of like, ah, okay, I, I kind of get how that consent hangs together. Or I kind of get like where some problems might be really quickly. Yeah, like you say, six months or bets are off.

[01:04:12] Matt: I don’t know what they’ll be doing and what they’ll be able to do. 

[01:04:15] Roman: My favorite part of using AI is when you, when once you use it like a bot for like 10 hours or something right? You start to get what, where it gets and what it can do and start using it for more and more and more and more you connect to things like Gemini, CLI is a game changer.

[01:04:31] Roman: And then at some point you’re like, I’m a very lazy person by nature. You know, like if you need to do a very complex thing, you need to ask a lazy person because they will find the easiest way to do it and the way that you don’t need to redo it again. So now I ask AI to write a context for other things that AI does and my favorite is go there to Google advanced consent mode, read the documentation, do research, make a page that’s going to explain everything that it does.

[01:04:54] Roman: If you talk to this in a new channel and say this is the documentation, we go to consent mode. Now build me the story for this and this and this. And because it gives the context, like you give it the context of everything. It doesn’t guess anymore. Like it still gets ’cause it’s a probabilistic model.

[01:05:08] Roman: But you can do pretty much everything. Yeah. It’s, it’s, it’s mind blowing. Good. 

[01:05:14] Dara: You, you’re not lazy. You, you’re not lazy. You’re efficient. 

[01:05:19] Roman: Yes. 

[01:05:21] Matt: Yeah. I need to play with something. We’re, we’re, we’re a bit, we’re a bit of a clawed code, contingent, measure lab at the minute, but we, we do like to mix your milk, try the different CLIs tools because they are getting better and better.

[01:05:33] Matt: Really, really cool stuff. 

[01:05:36] Roman: I’m using cloud as well for other projects as well. It’s amazing. They have a difference in the context window at the moment, but to be fair, who knows it’s going to be in a year from now, like, it’s probably going to be a very different world. 

[01:05:48] Matt: Well, that’s, I mean, you’ve, you’ve absolutely nailed the segue into what my, my normal sort of final question is, which is what is, if you had to look in a crystal ball.

[01:05:57] Matt: What would be your predictions for me? I originally, when I first asked this question, said five years and realized that’s ridiculous. We might all be flowing in five years. Yeah. But like a year, two years, where do you, what do you see the future of marketing analytics or anything of you, if you wish to be so ambitious?

[01:06:18] Roman: I don’t think that our jobs will stop. I’m pretty, I’m, I’m, I’m not that person who thinks that we’re going to get replaced with a robot. I just don’t like the concept of a robot. I feel like it’s an exoskeleton and just improves what you’re doing by tenfold. I think getting the data and just reducing things, like moving things forward is going to be a million times faster.

[01:06:41] Roman: But somebody, somebody said, I’m not sure if it was, on your podcast or somewhere else I’ve read, if you stop it right now, it’s going to take another 10 years for people to catch up with what happened. And, we’re not stopping the progress. I think this is going to be the biggest challenge. I’m trying right now to educate people how to do prompt engineering.

[01:07:00] Roman: Where to start, like, starting with AI is very difficult. This is so cool that you have, like Looker for example. They now have Gemini built in and like, and like to do, do all of that thing. But it really becomes an amazing tool only when you start understanding how to structure your prompt and how to work with it.

[01:07:17] Roman: Essentially. It’s a different level of programming. Like when I started doing programming, I was doing c and c plus plus. I need to explain how to get data in and how to get the data out and all of that things. Now we have a Python -like import solution. That’s how Python works, right? And now we have, charge GPT and like import Python.

[01:07:37] Roman: And this means just a different level of obstruction, that it comes in there. 

[01:07:41] Matt: Yes. So, the ultimate high order language, right. Non high order programming language. I forget which way round it is, but it’s just literally just saying what you say, what you need and it will do it. 

[01:07:52] Dara: I like your optimism. I was going to say, I like your optimistic view. There is obviously a lot of scaremongering and some people are just, you know, talking e even this week, I think Sam Altman was telling, US Congress that, you know, certain industries are basically just going to disappear overnight. And there’s a lot of this kind of doom mongering around, people losing jobs.

[01:08:14] Dara: and you know, maybe some of that will happen, but I, I like your optimistic view where, you know, you’re, you’re, you’re, you’re basically saying that, you know, it’s, it’s technology that’s there to help us and if we can learn to use it properly, it’s going to reduce some of the repetitive tasks and just make the, the good quality work that we do even better.

[01:08:31] Roman: Yeah. And essentially a lot of things like this, right? 10 years ago, we all expected Uber to have driverless access. We’re still not there. I think that, a real, real bound like boundary when it might happen, what might happen is that we’re just going to, hit that physical world realization that to change something in the physical world, you need years.

[01:08:51] Roman: Like one of the things that can happen is that it becomes, as more people start using it, it’s very energy expensive, but you start to lose how much energy you can like, like you don’t have enough capacity to power it to use for everything. And then building another energy plant takes years.

[01:09:08] Roman: It’s not going to, like, you can’t go to church between like, can you add another like plan there? No, not yet. Oh, really? Exactly. And that, I think, is another very big example. Like a lot of things are probably going to stay unchanged, but some jobs are going to change a lot. Like I would imagine everything related is going to be very different.

[01:09:28] Roman: But, again, Python came in and coding became way, way easier and cheaper. So what happened? We now have spice of money coders in 10 times with mandated scientists instead of doing the opposite. 

[01:09:40] Matt: We’ve always got plumbing and electricians. If we need to really have a backup, just get a trade.

[01:09:45] Dara: Just need to learn one of those. I guess you built your wall behind you, so you, you, you’ve got some skills. I was just thinking as a final kind of parting note, we need to, Matthew, what we need to do is take everybody’s answer to that question that you give. And in the future, which now isn’t going to be five years, it’s probably going to be five months, we need to give a prize to whoever had the best, best prediction.

[01:10:08] Dara: But you know, in, in, in all reality, it’ll probably all change in five weeks and we won’t even have a podcast anymore. Yeah, that’d be, yeah. We’ll just be agents, our voices. Yeah. Speaking to other AI agents. 

[01:10:21] Roman: Yeah. Talking about the podcast. My last, like the last thing that I found very fun is you can go to Notebook Lamb.

[01:10:28] Roman: You did all the preparation for a very complex topic and. Can you generate a five minute conversation about it? So a person that doesn’t know anything about it can listen to the background and have more incentive to go into the topic and help me to solve it. 

[01:10:40] Dara: So yeah, we’ve played around with that. Did Matthew, didn’t you put a, didn’t you put a GTM container into it or something like that?

[01:10:45] Dara: And it made a podcast and it sounded really like, they were really excited and they’re like, wow, on today’s episode we’re talking about GTM container 1 2 3 dash seven or whatever it is quite weird. Yeah. 

[01:10:56] Matt: I uploaded, I uploaded the j, the JSON export of a Google Tag Manager container and put it into Notebook lm, and it produced a podcast around it.

[01:11:05] Matt: It was quite, yeah, it was quite good, listen, it got more chemistry than Dara and I, the two virtual, it’s not hard voices. 

[01:11:11] Roman: No, I mean, I love your podcast. 

[01:11:14] Dara: Oh, well, well, thank you for saying that, Roman, and also because you, yeah, there were several other things that we could have talked to you about. I think.

[01:11:20] Dara: We’ll, we’ll, we’ll get you back on again if you’re, if you’re happy to join us again and, and we’ll cover another, another topic and we can look at how your predictions fared as well, maybe. Yeah, always happy to. Thank you very much. Brilliant. And, and look, thank you again for joining us. It’s been a great chat.

[01:11:37] Dara: That’s it for this week’s episode of the Measure Pod. We hope you enjoyed it and picked up something useful along the way. If you haven’t already, make sure to subscribe on whatever platform you’re listening on so you don’t miss future episodes. 

[01:11:48] Matt: And if you’re enjoying the show, we’d really appreciate it if you left us a quick review. It really helps more people discover the pod and keeps us motivated to bring back more. So thanks for listening and we’ll catch you next time.

• Spotify
• YouTube
• RSS Feed