How Skewed Analytics Leads to Bad Marketing

[00:00:00] Matt Wilkinson: The kind of worst issue I've seen from bot related activity my time working in digital was auto retailer we used to work with. I think we were all about to leave the office, you know, maybe it was a Thursday and we were going out for a drink or something. Everyone was on a bit of a high, and then suddenly we got a call from the client and, you know, all hell had broken loose.
[00:00:20] Cyril Noel-Tagoe: Hello everyone and welcome to Cybersecurity Sessions, our regular podcast exploring all things Cybersecurity. I'm your host Cyril Noel-Tegoe, principal security researcher at Netacea, the world's first fully agentless bot management product. With the busiest e-commerce period of the year fast approaching, companies are heavily relying on analytics to optimize their marketing strategies. However, in our recent bot management review report, we revealed that in addition to stealing user accounts and committing fraud, bots can actually have an effect on these

[00:00:00] Matt Wilkinson: The kind of worst issue I've seen from bot related activity my time working in digital was auto retailer we used to work with. I think we were all about to leave the office, you know, maybe it was a Thursday and we were going out for a drink or something. Everyone was on a bit of a high, and then suddenly we got a call from the client and, you know, all hell had broken loose.
[00:00:20] Cyril Noel-Tagoe: Hello everyone and welcome to Cybersecurity Sessions, our regular podcast exploring all things Cybersecurity. I'm your host Cyril Noel-Tegoe, principal security researcher at Netacea, the world's first fully agentless bot management product. With the busiest e-commerce period of the year fast approaching, companies are heavily relying on analytics to optimize their marketing strategies. However, in our recent bot management review report, we revealed that in addition to stealing user accounts and committing fraud, bots can actually have an effect on these analytics by creating poor quality data. And to explore more about how bots can affect marketing analytics, I'm pleased to be welcoming my special guest today, Matt Wilkinson, who is the head of Paid media at Spike Digital. So, hey Matt. Thanks for joining us today.
[00:01:06] Matt Wilkinson: Hey, thanks for having me along.
[00:01:09] Cyril Noel-Tagoe: Before we get started, would you like to quickly introduce yourself to our listeners?
[00:01:14] Matt Wilkinson: Yeah. So, my role here at Spike Media is to kind of head up the paid media accounts, so, you know, across search and shopping, a kind of a, sort of pretty broad remit of clients. So we kind of have quite a lot of e-commerce clients that we look after. So obviously a really kind of key period for them coming into Black Friday. But you know, even for the lead gen type clients, you know, it's a time of kind of increased traffic, increased competition, so, you know, it's something we need to keep a close eye on across the board really.
[00:01:51] Cyril Noel-Tagoe: And how does this peak e-commerce season, the Black Friday and the holiday season affect marketing activities?
[00:02:02] Matt Wilkinson: Probably the assumption is, you know, Black Fridays are sort of bumper time for, you know, kind of anyone operating in the online space. But it's very much a double-edged sword from my experience. So, yes, you know, a lot of retailers are going to see increases in sales, kind of spikes in conversion rates, that kind of thing. But not everyone is the perfect fit for Black Friday. So, you know, not all products are the kind of the frontline exciting products that people are looking for on Black Friday. Not all sites, not all clients are kind of the best fit. So, you know, quite often what you will see as well is that increasing competition, particularly in the kind of display and social space, that extra increase in competition is actually a kind of hindrance rather than a help to a lot of advertisers. So, you know, it just means there's more people fighting for that same advertising space and driving up costs and, probably with more enticing, more relevant offers that are going to really, really pull consumers in around Black Friday. So, you know, it can be quite a challenging time for a lot of advertisers as well.
[00:03:13] Cyril Noel-Tagoe: I guess, and with that in mind, what's your typical advice to your clients around this period?
[00:03:18] Matt Wilkinson: So, I think again, it sort of depends very much on where they sort of sit on that spectrum. You know, if they're a perfect fit if you like for that sort of Black Friday buzz or, if they're on the opposite end of the spectrum. But, you know, certainly for those that are retail focused, make sure that you're ready really for that surge in traffic. So, you know, make sure that your site is set up to be able to handle that extra traffic. Make sure your budgets are ready, and your teams are ready to respond. That your stock levels are where they need to be or all of those things really. But yeah, on the other side of things, be prepared for rising costs. Be prepared for a spike in CPMs or, cost per clicks or whatever that might be. Potentially be prepared for a drop in click through rates. If, as I say, you maybe not the most kind of appealing type of business in that Black Friday space, it might be a difficult time. Certainly there's a number of clients that I've worked with where the best strategy for them is to simply turn off their marking activity from maybe the week before Black Friday and wait until all of the cyber buzz is out the way and come back on a few days after that actually that's the most efficient approach for them to just, to avoid burning a load of budget on stuff that's not going to work as well.
[00:04:49] Cyril Noel-Tagoe: So we did a report. We released a bot management review, which looked at how businesses have dealt with bot attacks over the last year and especially coming to Black Friday. I think this is quite interesting to look at because when we did this, we surveyed over 400 businesses across the UK, the US, large enterprise businesses. And the vast majority of them have told us that bot activity had impacted their analytics and created poor quality data. And not only that, but that had actually caused a financial impact to them. I think 60% of them said it was a minor financial impact, and 27% of them said that was a moderate financial impact. I was just wondering, is this something you've come across or experienced with your clients?
[00:05:34] Matt Wilkinson: I guess a large part of this is, that visibility and awareness piece, isn't it? I think a big part of the bot issue, the bot challenge is the bit of the iceberg that's under the water. I'm sure over the years, both myself and clients that I've worked with have been blissfully and unaware of a lot of that. But in terms of what I have seen, and I am aware of myself, I mean it's, it's very rare I'd say that you'd go into, let's say a client's Google Analytics account and have a look around the data and do a bit of analysis and, if you're looking over a decent period of time and you dig in deep enough, I'd say it's unlikely that you wouldn't find some kind of weird data anomalies in there of large amounts of traffic from unusual locations or spikes in traffic over particular times from, I dunno, a certain device or something like that.
[00:06:29] Cyril Noel-Tagoe: I guess it's interesting because there's probably even more beneath the surface that they aren't seeing. But the way I look at it, I see two main ways that bots can impact analytics. And the first one, the intentional kind of ad fraud type bots, which are there to generate impressions, views, whatever, to exhaust those days' budgets. But I guess the more, sinister ones is the unintentional ones. So these are just not normal bots, like maybe a scraper bot or a scalper bot that's there for a specific purpose. That purpose isn't to skew the analytics, but because it's there and it's active in a non-human way, it's actually polluting that data set. And I guess in some instances you'll be able to see there's a big spike from a location that you don't expect, but other times it might be so ingrained in your traffic that you might not actually be able to identify that over long periods of time just because there's constant activity. Is that something you've discussed with your clients before?
[00:07:35] Matt Wilkinson: A lot of this stuff is under the surface. But you know, also I think there's a bit of a perception that, if you take Google Ads for example, where a lot of our clients and most clients' money is spent. They have their own processes, their own kind of procedures for filtering out bot clicks, fraudulent clicks and essentially refunding the cost for those clicks. So I guess on the face of it, what's the main reason maybe to care about that bot traffic? Because it's costing us money. It's not costing us money because Google's finding it and giving us that money back. So problem solved. But there's a couple of issues there. One is, how confident can we be that Google is able to find and filter all of that stuff out? Well, three issues really. Because, certainly from my experience, Google is a lot hotter on this than the vast majority of other places where we might be spending money on media. So, how well are Facebook and LinkedIn and Bing and, certainly my sense is that their processes aren't as robust as Google's, but even beyond that. Okay. So we might not be paying for it even if we assume that all of that is being credited back to us. What's that doing to the data? So, we're getting the money that's being spent on those clicks that's coming back, but that traffic isn't being filtered out. All of those different data points that that traffic will still be probably feeding into your analytics data, which is then feeding back into your audience data, which is maybe a big part of what you're using for your targeting and driving your results. Certainly an increasingly important area of digital is all of that kind of audience led side, the data driven side that really, really leans on, how good is your data? How valid is that information that's powering all of these bidding and targeting decisions. It's quite hard to put value on that impact.
[00:09:47] Cyril Noel-Tagoe: Yeah. And I guess, break it down for me a little bit more in terms of, kind of that analytics piece. Obviously you've got the stuff that's coming from stuff like your Google Analytics, but also you've got your own data. How often is that kind of merged together for organizations?
[00:10:02] Matt Wilkinson: I guess, probably like a few angles to it. So the in the kind of, sort of more old school approach, if you like. You know, all of that kind of account optimization and kind of targeting and bid management and all of that kind of thing. You know, obviously going back, I dunno, probably quite a way now, actually, maybe going back sort of like 10, 15 years even. All of that would've been done manually. So, you know, in the midst of, let's say a Black Friday period when the kind of sales are coming in thick and fast and there's somebody there managing the account, trying to kind of squeeze the best value out of it. Those decisions are gonna be made on that data that's coming in. And you know, that's certainly, that's still the case. But what's happening now is that that data is, you know, it's real time. It's as that data comes in that's kind of powering the machine learning, that's powering all of the kind of algorithms that are then making these decisions on the fly. So I, you know, I guess back in the kind of steam powered age of where all of these things were done by a person sat tapping away at a computer, that those impacts would've been felt much more slowly. Whereas, if all of that is kind of being automated and, you know, algorithmically driven and it's kind of, you know, dynamically feeding into these audiences, you know, it's happening so fast. And to what extent are those odd behavioral patterns being picked up and being filtered out, you know, and if they're not, at what point is someone then gonna pick that up and respond to it, but at what point is there gonna be that sort of human intervention, I guess.
[00:11:46] Cyril Noel-Tagoe: Yeah. And I guess because you've just got such a pointed kind of, business peak period and everything is kind of pointing at this one period, you don't really have the time to kind of have that retrospective look over it, where if it was kind of a longer period, you could say, "Actually yes", look back and see how this is affecting the change strategies. But for Black Friday especially where kind of this one weekend, you could just go this, be relying so much on the, on the automation and that. But, in terms of kind of quantifying, I think you mentioned earlier, like how do we quantify this? So one of the questions we asked in the survey was for them to try and estimate what the skewed analytics from bots was costing them as a percentage of their online revenue. And for these organizations, kind of the online revenue was about, on average 50% of their total turnover. And on average, what the businesses were reporting back to us was that these skewed analytics was costing them 5% of their online revenue, which it seems like a very sizeable chunk.
[00:12:53] Matt Wilkinson: Wow. I mean, yeah, 5% to put that into context, if you imagine a really kind of mature media account, you know, whether that be kind of in the display space or in the search space, if you could get yourself a 5% increase in ROI, in lead volumes or whatever that might be, that that will be huge. So, yeah, to suggest that, that's the impact that's happening is, yeah. It's pretty significant.
[00:13:24] Cyril Noel-Tagoe: Yes. I mean that 5% is over the course of the year. I'm just trying to understand, if we look at Black Friday specifically, how much more of an impact, how much more is that 5% felt over Black Friday versus kind of the rest of the year?
[00:13:40] Matt Wilkinson: Yeah, yeah, yeah. So I mean, I think there's a number of reasons why that impact is gonna be more extreme over Black Friday. Any ground lost over that period is gonna hurt twice as much. A lot of, well, I guess most retailers are bringing in the vast majority of their revenue for the year is coming in q4. So, you know, Black Friday, Cyber Monday, you know, has now become a pretty significant slice of that Q4 impact. So yeah, I mean, if you're seeing 5% loss there, then? Yeah, I mean, I guess it hurts just as much as 5% any other time of year, but if it, I guess that 5% can quickly get amplified by, if that's when you run in promotions, if that's when margins are particularly tight, you know, if that's when you've got challenges around stock. I guess pretty quickly that that 5% can turn into a bigger problem due to the impact of some of those other factors as well. So yeah. Really, really significant.
[00:14:50] Cyril Noel-Tagoe: Yeah. And just taking some of those other factors there. So you mentioned, for example, kind of like stock issues and one of the things that bots have done, especially over kind of the last few years where we've had like this, the scalper bots that are gonna buy up inventory very, very quickly. And then obviously you've got almost, there's this bot that's buying up inventory, which is causing you a stock issue. And at the same time, just the activity of that stock is causing you this marketing, this analytics issue at the same time, and it's just this double-edged sword that you're facing at a very important time for you.
[00:15:25] Matt Wilkinson: I guess there's all kinds of things that can go wrong and that's why it's such an intense period for anyone working in online, but, you know, particularly in the retail space. The kind of worst issue I've seen from bot related activity over my time working in digital was an auto retailer that we used to work with, and they just, one kind of random evening. I think we were all about to leave the office. Everyone was on it, you know, maybe it was a Thursday and we were going out for a drink or something. Everyone was on a bit of a high, and then suddenly we got a call from the client and, you know, all hell had broken loose. And essentially what had happened is, well initially they had no idea what had happened. They were basically seeing tons, tons and tons of calls coming through on their sales numbers. And basically like, the line just couldn't handle it and had been wiped out. And as it turned out, it was basically a denial-of-service attack on a number of their phone numbers to the point that it did essentially wipe out a big part of the business that evening and created all kinds of chaos, which, you know, thankfully was sort of resolved in a few hours. I don't even wanna imagine the kind of damage that something like that could do to a retailer if that landed around a peak period like Black Friday, Cyber Monday or something like that.
[00:17:00] Cyril Noel-Tagoe: Yeah, and I guess it doesn't even have to be... so in that case, that sounds like a very intentional denial of service attack. But sometimes it doesn't even need to be that, right? It can just be, you've already got this increased load on your website of a Black Friday, and then you've got the, whether the bot is there to cause a denial service or it's just there for another purpose, whether it's a scalper bot or scraper bot, whatever. It's also gonna be adding to that load, and that is just, I think some researchers kind of estimated bot traffic around Black Friday last year was about 35% of the total website traffic so that we've already got that massive increase in volume at an already peak period. And I mean, we did some consumer research last year as well around Black Friday. Just talking to consumers. And I think it was about 58% of them had experienced kind of technical issues going on websites, whether that's slow loading or the actual website completely crashing. And obviously once you've got these bots causing this extra load, you can kind of understand why that's happening. And then, I think another 32% of them were saying that they couldn't purchase items they wanted due to them selling out quickly. Also when you've got scalper bots, that could be part of the reason there. So, I mean, that's just some of the research we've done.
[00:18:22] Matt Wilkinson: So what's making up that 35% in your kind of experience? Like what types of bots are drive in the majority of...?
[00:18:32] Cyril Noel-Tagoe: So it really depends on the type of site really. For e-commerce sites, you're gonna have scraper bots, which are trying to do price analysis, especially for competitors or other sites. But then you've got, especially where you've got kind of hype items around Black Friday, where you've got some items that are high in demand, low in stock. You can have scalper bots, who wanna buy it up and then put those onto reseller sites. So kind of, you've already got that going. You might have intentional denial of service attacks as well. But you can take a competitor down because, as you said at the start there, there's only so many outlets and so many consumers, if they can't get what they want from your site, they're gonna go to the next one, and get that traffic. So yeah, it's a mix of things, but I think just because of the intense kind of period of everyone's coming to shop and we need to make sure that as you said, like this accounts with so much of our forecast for the year, and just these attacks can be absolutely intensified. Coming on to preventative measures then, I guess in your opinion, in your line of work, what do you think we can do to kind of educate businesses under risks of parts, like to analytics, but also kind of these other risks we talk about? The impact on the website load, their customer satisfaction, things like that?
[00:19:58] Matt Wilkinson: I mean, I think to be honest, just getting some of these numbers out there and more visible. I think they speak for themselves really. As I was saying before, I think that maybe that 5% needs a bit more context because you hear 5% of anything, it doesn't seem like that big a deal, but, if a sizable business is losing 5% of its revenue, that's a pretty big deal. And the 35% of traffic around Black Friday. I mean, you know, these are a kind of numbers that become quite hard to ignore. So, I think by increasing awareness of some of those hard facts I think that that does a really good job of focusing people's attention on the issue. I guess in terms of preventative, that sort of becomes a bit more challenging. I suppose from my experience it is more about after the fact. So, you know, it's more about identifying, there was kind of weird anomalies in the data and, you know, filtering that out of reporting or, you know, maybe finding ways to strip that out of any kind of audience builds or anything that might be fueling bid strategies and that kind of thing. But, I mean, I suppose the true preventative side, that falls in your court really doesn't it, in terms of how to kind of tackle it before it happens.
[00:21:27] Cyril Noel-Tagoe: Yeah, and one of the ways I love to emphasize how significant that 5% is, is to relay it back to something like GDPR, which came out and that the maximum fines there were 2% or 4%. And they're the absolute pressure that's put on businesses, like you saw massive transformation programs for that 2% or 4%, and now here you've got a potential 5%, which is even more of that. I think that's really puts into context just how much of an impact this could be. But yeah, in terms of the preventative, I think one of the really important things is for marketing and security departments to speak to each other because on one side you might... Security who see there's a bot problem here, but they're not communicating with marketing. So marketing are still relying on that data. Or conversely, a marketer might see, "Oh, there's this spike here that doesn't look right to us. This data seems different." And then they can relay that back to security as well. So I think you definitely need to have that constant communication and those channels between the two. I mean, I'd be remiss if I didn't say, obviously get proper bot management tool in place. But absolutely. It'd be really interesting getting your perspective on this. But before we wrap up, are there any last thoughts you'd like to share?
[00:22:56] Matt Wilkinson: I think just everybody have a great and a safe Black Friday. And watch your bikes out there.
[00:23:06] Cyril Noel-Tagoe: Great. Thank you so much Matt. And thank you to all of our listeners for tuning into this episode of Cybersecurity Sessions. If you've enjoyed this podcast, please be sure to subscribe and like, or leave a review on your podcast platform of choice, and we'd love to get your feedback. And you can also get in touch with us via Twitter @cybersecpod or by email to podcast@netacea.com. Thanks again for listening. Have a great Black Friday and see you again next month.