AI in Cybersecurity: A Double-Edged Sword – Elaine Lee, Mimecast

[00:00:00] Elaine Lee: Deep fakes are really just made possible by all this widely available data that's being recorded. A cloned voice sample can be developed with four seconds of footage, which is pretty amazing when you think about it. I mean, just me was probably four seconds already.

[00:00:17] Andy Still: Good day, everyone. And welcome here. We are again for the latest installment of the cybersecurity sessions, our regular podcasts, talking about all things cybersecurity with myself Andy Still CTO and founder of Netacea the world's first fully agent must bot management product.

So today we're talking about email, a tool that we all rely on. But also one good historically been the entry point for many cyber attacks, malware distribution to phishing attacks and with the growth of AI, these attacks again, evermore sophisticated, we're lucky today to be joined by Elaine Lee data scientists with Mimecast who can tell us some more about how AI has been harnessed as an attack vector, but also as a means of defense.

So welcome Elaine. Great pleasure to talk to you today.

[00:00:59] Elaine Lee: Thank you, Andy. happy to be on the, talk today. ,

[00:01:02] Andy Still: Before we start,

could you quickly introduce yourself for our listeners?

[00:01:06] Elaine Lee: my name's Elaine and I'm a data scientist at Mimecast, primarily focused on incorporating AI and machine learning into email security products. this is actually my first stint in the cybersecurity field and I'm really enjoying the dynamic aspect of it.

Um, especially since there's a adversarial and fast moving component That keeps us on our toes really. previously I have worked in healthcare and FinTech and the federal government. So Yeah.

happy to be here and to, share my knowledge about this.

[00:01:35] Andy Still: Thank you very much, Elaine. so I think everywhere we turn at the moment, there seems to be talk of new problems being solved with AI. So it was that opening question is AI taking over the world?

[00:01:46] Elaine Lee: Is it taking over the world? Well,

we're definitely seeing more of it. That is true. there has been an accelerated adoption. Thanks to. the performance improvements of hardware, just leaps and bounds over the last decade. hardware is getting more powerful. Storage is getting cheaper.

Compute power is getting faster and, Also cheaper. so this hardware, advancement has really facilitated some of the growth. And also we have a lot of, experts and practitioners who have built a lot of commercial off the shelf tools that allow anyone to basically incorporate some form of AI their products.

[00:02:22] Andy Still: Yeah, I think It has definitely moved out of the, regime of pure academic or very sophisticated expertise into much more mainstream. So you can with minimum kind of knowledge, go on to. On demand systems such as AWS, et cetera, and easily get up and running with image recognition or something that previously was the domain of sophisticated AI.

Those kinds of things, like you say, they are an absolute game changer. I think my, other question around a lot of the AI systems. speaking from the background of, having built a complex AI system. And see our product go head to head with other products claim to have AI.

how real do you think a lot of the claims of things be true intelligence actually are.

[00:03:04] Elaine Lee: that's a good question. I think it's helpful to distinguish. these buzzwords that we're seeing a lot more of, especially in product documentations and marketing materials, AI specifically has actually been around for a long time. its core, the definition of AI is Really a system that contains a rules or instructions, that instruct a computer, how to perform a task. So depending on what it is, Even just a simple Computer program that people have been writing for the last 40 years that could fall under the classification of AI.

If it's just instructing the computer to perform a specific task, often like a categorization task, for example. Um, so broadly speaking, a lot of things could be defined as AI. Machine learning is. I like to think of it as a subset of AI. So machine learning is less about receiving a well-defined set of instructions on how to perform a task.

And more about, uh, receiving a large set of examples to learn from, to learn characteristics about, basically to infer patterns about, and then using those inferred patterns to perform the task itself. So That's the difference between machine learning and artificial intelligence and, regarding these marketing materials that companies are putting out about their products continuing AI. Yes. I believe that's a relatively low bar to achieve in this day and age. So that's not relatively groundbreaking or informative, uh, in terms of, product capable. Um, but if they do mention machine learning, that might be worth paying attention to, there might be something special going on there.

[00:04:45] Andy Still: Yeah. And the, the other, buzzwords you have as well as deep learning as well. what's your take on the difference between machine learning and deep learning?

[00:04:53] Elaine Lee: Deep learning is a type of machine learning that borrowers inspiration from biology, specifically neuroscience. In fact, deep learning used to be more commonly known as artificial neural networks. And as you can guess from the name neural network, it is mimicking the behavior of the human brain and how the human brain learns.

Now why is it called deep learning? What does deep actually refer to exactly? the deep refers to, the different layers of perception that is in a deep learning system. So kind of mimicking how a human brain works when it observes something in front of itself. it doesn't notice everything at the same time.

For example, if I saw an animal in front of me, I might notice first how big it is, whether it's large, small medium-sized, that's like the first thing I would notice. And then the second thing I would notice is probably its skin, whether it's very smooth or scale. Et cetera, et cetera. And then I might notice some other details about its face or its tail.

And you know, all this stuff, is perceived gradually. And the deep learning system mimics this behavior. And. By learning things gradually. That is, uh, so for example, one of the first popular, deep learning models was actually built to detect handwritten digits. This, as you can imagine, uh, was very useful and practical for the postal service, which was still processing mail, with handwritten digits on the envelopes representing the zip codes.

So the, if you were to peek under the hood, Deep learning model. You may see a layer that corresponds to I'm looking for a horizontal, straight lines that go left to, right. So some digits we'll have this feature represented very strongly, such as the number five with it's a little hat at the top of the digit.

the next layer may be looking for a very straight and vertical lines that go top to bottom. So some digits that might exhibit this feature strongly are the number nine, for example, or the number seven, and then the third layer. Maybe looking for a curves in the digits. So, uh, digits that have this feature very strongly, as you can imagine is the number eight and also the number two to a lesser degree.

So a deep learning system or an artificial neural network that is trained to identify digits, , perceive the features of the digits in this. as it relates to, uh, cybersecurity, I just gave two visual examples, images of digits, and also, uh, you know, a very visual perception of animals in front of me.

so as you can imagine, this can be applied to, you know, anything that's visual images, video, et cetera. and other common application is being applied to typed or written text human text. If you will. A deep learning system can pick up on attributes about the text in question.

So in a nutshell, that is what deep learning is.

[00:07:50] Andy Still: Yeah, I think we're looking at the power of AI and I think, you know, there's, a view of artificial intelligence and. Intelligence is, a very wide ranging word.

But what artificial intelligence does is it allows specific tasks to be solved very intelligently beyond a set of simple instruction. So it can learn how to do that in a, in a relatively nuanced way, but they tend to be very focused, very specific, problems that solve and the thing that.

Segues quite nicely into talking about your experience in cybersecurity. and particularly around some of the challenges on email security is as you're looking at it now, go into a bit more detail about the kind of AI approaches that you're seeing to evolving threats in the email area.

[00:08:34] Elaine Lee: So, attackers are definitely incorporating more, AI into their attacks. That's for sure. one strategy I could think of that's a bit more nuanced is, basically gathering vast amounts of information about the target in question, , this usually informs a sophisticated social engineering type attacks where they definitely have to do a lot of research and invest in data collection on the target first. And then secondly, they use that information to, craft a social engineering attack that is.

Likely to, entrap the target in question. Machine learning enables them to craft a more convincing attack. I mean, the attack would have happened regardless, but it may have happened with less finesse. If they did not incorporate machine learning or artificial Intel, Into it, you know, before it would just be a less finessed attack, maybe it would just be an email from some random person like the Nigerian prince. If you will, asking you for money, that's like a not very finesse attack, Um, that's the before and now after with machine learning capabilities. What have you just access to more data about people, about the target?

You can figure out who their CEO is. That's like an easy one, but you know, that's made possible by the data that's now available online, such as LinkedIn, you could create a more targeted attack that way by instead of pretending to be a Nigerian prince, you pretend to be the CEO and then maybe a step further.

If you get your hands on the information, you can pretend to be their direct manager. And then crafting the attack such that the sender seems to have a close relationship with the target makes those attacks more convincing and the subject more likely to fall for it. So, yeah, this is all possible based on the availability of information with some machines. Techniques, uh, that can be applied. They can identify who's close to you who is close to the target and then impersonate that, person. , so the attack has a greater likelihood of succeeding.

[00:10:33] Andy Still: back in the past, you would have kind of phishing attacks, which will be very. Kind of scatter gone. They would just fire off, , tens of thousands of emails without intelligence. Then you might have spear phishing attacks where there would be a kind of human involvement to , target a particular person, learn more about that person, their potential weaknesses.

From what you're saying, it sounds like what the use of AI has done is allow the spear phishing attack to be much wider and much closer to efficient attack, because they can gather that data. They can imply that human, like intelligence to that data to make it a targeted attack. So you get in the benefits of the spear phishing attack, but with the effort of a phishing attack,

[00:11:15] Elaine Lee: Yup. I could give another example, of how machine learning has enabled attackers, uh, in a way. So this one's a little bit more nuanced and it involves, uh, the attacker suspecting that the target is running some sort of machine learning model, .

Or even just AI defense, some sort of defense system at the, target site. And then the attack strategy then becomes how do I trick that defense system to lower its guard so my email has a greater, chance of landing in the.

And then from there, you know, once you're in the gate, then, if the user clicks on it, then the rest is history. Right. So, um, there's that sort of attack basically AI versus AI, right. So it's trying to fight the, Defense system and, depending on what it is, it's, uh, yeah, they could use AI or a bunch of automation to provide us the defense system and get it to lower its guard to trick the system in some ways.

[00:12:11] Andy Still: a lot of these techniques are just about, getting people to lower their guard, to, to put their trust in, in what they're seeing. I know one of the things that was mentioned. And I think this is some research that that man cast have done was around the use of deep fakes as part of phishing attacks.

Could you just share some more about, how deep fakes are used as part of this.

[00:12:31] Elaine Lee: Deep fakes are really just made possible by all this, , widely available data that's being recorded. such as this conversation, for example. Um, so you know, all, all this data, all this audio and video. And image footage is out there on the internet now, and it's getting easier and easier to find it and to categorize it.

And just like, maybe it's easier to find recordings of you Andy online. And I know that it is actually you online and to just create these convincing thought, like entities. That can go about and pretend to be you and just do all sorts of awful things. Um, yeah, it's made possible by all this.

available data and that does give rise to deep fakes, , at its most basic, it could just be the splicing together of, of using various words and just like making a coherent sentence that way that is at its most basic, obviously, you know, it's probably not going to sound very good, but with the, again, back to the whole, Advancement in computing, reduction in cost of computing technologies, powerful computers that are available to people. Um,

you can use all these fancy audio and video processing features to just smooth out that content, that faked content and deliver it to. To the targets in question. we did say in one of our articles that mind cast put out specifically the one in intelligence CA so back in 2019, we did say that a cloned voice sample can be developed with four seconds of footage, which is pretty amazing.

When you think about it. I mean, just me was probably four seconds

already.

[00:14:27] Andy Still: I mean, I think it's, when you, talking before about getting people to lower their. Against things. One of the things that people have inherently always trusted with, they could actually hear the voice of the person. , if you got the email telling you to transfer, thousands of dollars to another account, you would doubt it.

But if you got a voice message from the person again, that's usually crossed another way. So I think it's just constantly looking at. Getting people to lower their guard. And then the idea that these could be scouring the internet right now to get appropriate voice signals and almost everyone is recording something and putting, putting some kind of video out there, particularly senior, members of C-suites of most companies of.

Uh, presence out there. So the idea that it's as little as four seconds, that it takes to, to be able to do that again,

just need to be aware that this is something that can be happening and adjust your processes around it. But awareness of that, think it's very, fascinating sort of how easily these things can be generated.

[00:15:23] Elaine Lee: I agree.

[00:15:24] Andy Still: I think we've talked a lot about how AI is being used for Barden, how AI is, , targeting bypassing defenses on emails. , your day is spent actually trying to protect our emails. So, how are you actually using AI as part of your cybersecurity defense approach?

[00:15:41] Elaine Lee: Sure. , in summary, The best way that AI helps out, uh, from a defense standpoint is, , we just have to play to the strengths of AI. So AI, systems, they are computers after all. Uh, they are very good at processing vast amounts of data. , they're good at remembering things that humans can't, you know, humans can't remember everything, but, , AI systems can remember things pretty well.

They're also very good at, Processing a bunch of information at the same time. So , that combination makes them very good. At anomaly detection so they can pick up on things, deviations from normal behavior of much, much more easily than a human could. So a lot of strategies are centered around that sort of theme. So as a result, , a lot of AI systems that us, in the cybersecurity world build are uh, are centered around. What's weird. What's let's, let's try to, let's build a system that's very good at alerting when things are a little weird and let's tell it to look at these sorts of characteristics.

And if there's any deviations in these sorts of characteristics, red flag and get a human to look at it. so that's high level. That's how the AI helps us from a different standpoint. .

[00:16:52] Andy Still: So would you say that AI was now a fundamental part of your, defense strategy?

[00:16:58] Elaine Lee: Yes. Yes it is. and we have, uh, various products. Basically play off of that theme. I just described, these products are all situated in various parts of the security defense systems. Um, For example, I primarily work on inbound emails, uh, going to our customers, users, uh, just building the.

Systems around that. analyzing content and communication patterns between our customers, users, and the senders of the emails. Just simply looking at communication patterns, a little bit of content analysis. That's just me specifically. There are others at the organization who do a deeper analysis on.

Contents of the emails and they built machine learning and AI models around, content analysis. Looking at attachments, , identifying your ELLs, embedded with an emails as potential risks or potentially safe, there's a lot of, , AI work going on across Mimecast.

[00:17:52] Andy Still: Yeah. So , I'm thinking about the kind of phishing attacks I'm thinking particularly of the kind of emails sent to, , people in finance departments asking them to make payments claiming to be on behalf of the CEO is the sort of thing you're looking at there.

Even down to the actual kind of language that is used in those emails. are you training those based on known attack factors or are you training those on looking at typical content that that user would be looking at? Oh, is it a combination of those.

[00:18:18] Elaine Lee: Definitely a combination of those.

we do have, specific teams that are dedicated to, research. Unifying and incorporating knowledge about known attack vectors into the systems. , there's definitely that aspect of the work going on, but, in order to be agile and to, respond to novel attack types, uh, we do have to, uh, look at, machine learning and AI, to.

incorporate those components into the system too. Again, going back to the anomaly detection type of theme. If they see something unusual, flag it, maybe it's a false positive, but at least flag it. And then if it ended up being actually malicious, then that's a good thing that we flagged it.

the AI and machine learning system is definitely crucial for identifying things that are never seen before. Whereas these known attack vectors. They are known for a reason they've been seen before. So, Yeah.

so we definitely have to use a combination of.

[00:19:10] Andy Still: Okay. I think of the things, whenever I've looked at, AI systems AI is good for solving certain problems. Humans are good for solving certain problems. Um, the combination of humans plus AI, he's usually the way to go. is that kind of reflect.

few of the world as well, does it.

[00:19:26] Elaine Lee: Yes. I, I totally agree with that. humans definitely need to be working closely with the AI systems and to, you know, Very involved in the development of these AI systems. we definitely need that human in the loop because I think it kind of does go back to something we spoke about earlier in the conversation about AI taking over the world.

there seems to be that perception. Yes. But, in order to prevent that from truly happening, we definitely need the humans to stay closely involved and to monitor the AI systems. Make sure they don't, adapt to quickly and into and in bad ways. , honestly, a little joke that I like to make is if you've ever written a program and it infinitely looped.

Yeah. You know, you know what that's like, that's just, that's a system running a muck and, uh, AI systems also can do something similar where, it.

could just get into its own little local Maxima or local minima. They just get stuck somewhere and then they just keep doing the same thing over and over again.

Um, you know, examples that we have seen in real life of an AI system, going a muck like that typically have been in recommendation systems. for example, , there were some criticisms about the YouTube recommendation algorithm, going down very dark paths, uh, shall we say, just the quality of the recommendations, getting kind of bizarre and strange and not, desirable.

So that's an example of, you know, , a system that needs maybe not so much direct human supervision, but definitely some safeguards engineered by humans. You don't just put into that system. That's why humans definitely need to work closely with the AI systems that they build.

[00:20:58] Andy Still: Yeah, absolutely agree with that. Like we said earlier, intelligence is a very nuance word and artificial intelligence tends to solve very specific problems, but it doesn't bring that human intuition where there, For want of a better word, common sense that you would get from having a human vet, your YouTube, recommendations.

And there's, there's plenty of other examples where you see problems that have been tried be solved with AI. And then humans have got involved and have managed to gain the AI to do something that clearly wasn't intended to do. Just before we wind up. Anything else that you would like to share with us today around the subject of AI in cybersecurity?

[00:21:36] Elaine Lee: Yes. Uh, yeah,

I would also like to share, uh, aI has been used to. , accelerate the sophistication of defense strategies. So we did talk a little bit about this already. Um, but something that I also want to point out that we do at Mimecast is, we're also using AI to.

Craft more convincing awareness training modules for our customers and users.

So, you know, this kind of

goes back

to the human and defense theme where humans can sometimes be the best, defense against these attacks.

So, uh, you know, a few months. ones, then we should make sure they have the information and the know-how to defend themselves.

actually awareness training is a pretty big part of our product and as the name implies, we craft these scenarios, these fake, but real looking emails that we deploy to the users at the customer site periodically. It's crafted to look very convincing and if they interact with it, then they're notified that they failed the exercise and taught, What happened here, this is what an attack could look like.

And, uh, in the future , to protect yourself against this attack, uh, use another method of verification of the content in your email. So, uh, these awareness training exercises it's not a novel concept. but We have started using AI and machine learning to make more nuanced scenarios, kind of.

You know, awareness training used to

be the Nigerian prince emails.

Um, but now it's more of, um, oh, let's let's craft a scenario where your direct manager is emailing you to Venmo them, uh, 200 bucks or something like that. So, yeah.

Awareness training. Even more important. Um, yeah, so it was just educating the humans and they are often the best defense, especially against zero day attacks or novel types of attacks.

So that's, we cannot underestimate how important humans are in this whole defense strategy.

[00:23:33] Andy Still: No, I think that's really good because I think humans are easily the weak point. What can also be the strong point? If you, if you appropriately training people? I mean, we've, I've been on the other side of those awareness attacks of having fake phishing emails sent to our, our address and.

Some of them are very convincing and having seen

examples of real phishing emails they have, you can absolutely understand why people are falling for them. The, you they play on known weaknesses. Like not wanting to question the boss or, you know, you say the amount seeming reasonable.

one thing which actually the, the deep fakes thing resonated with me, we we've seen. Recently, if people just requesting what's up numbers

from

people

and that presume that

is a way of, of them bypassing defenses, because things go straight to WhatsApp, which doesn't have the same kind of protection that we have around emails, things like that.

Don't know if that's something that you've, you've seen a rise in as well.

[00:24:29] Elaine Lee: well.

I have gotten a lot more spammy messages on WhatsApp recently, and I'm not entirely sure why, but Yeah.

that's

Yeah.

I've

definitely seen that a lot recently. And, um, Yeah, these attackers, if there's, yeah, there are so many different Media to reach people. So they're going to try everything I guess we just always have to just be vigilant. And if, and you know, us humans, we have a pretty good intuition of what's normal versus what's not normal of the people that we spend time with, including your coworkers. So if something seems a little off we should heat our intuition and

proceed with caution.

[00:25:04] Andy Still: yeah. Definitely use your, use your human intuition. Thank you very much for joining us today, , thank you everyone else from listening in. If you've got any feedback or. tweet to our Twitter account at cybersec pod. subscribe, leave a review. Any questions you want to use?

Good old fashioned email, which will be protected by Mimecast. Um, you can get to us at podcast at Netacea dot com. so thank you very much, Elaine, for joining us today.

[00:25:29] Elaine Lee: Thank you, Andy. Thank you for having me. This was a wonderful conversation and of course, if you're on the market for email security products, definitely check out mimecast@mimecast.com and we also have our own podcast called fishy business, which you can find on Spotify or wherever you listen to podcasts.

So again, thank you,

Andy. This was.

[00:25:49] Andy Still: Thank you very much, Elaine. And thank you everyone. And we will see you again in the next episode.