High-speed train hacks and homicidal lawnmowers • Graham Cluley

Smashing Security, episode 468: High-Speed Train Hacks and Homicidal Lawnmowers. With Graham Cluley and special guest Geoff White. Hello, hello, and welcome to Smashing Security.

Smashing Security, episode 468. My name’s Graham Cluley.

Geoff, welcome back to the show. Always a pleasure to have you on. Of course, our listeners know you well from your books, your podcasts.

The Lazarus Heist is probably the most famous one, isn’t it?

Which the BBC has renamed Cyberhack.

The problem we had was it was called The Lazarus Heist because, as some of your listeners will know, it’s about the Lazarus Group, the famous North Korean elite hacking team.

And so obviously the podcast was about that, but the BBC and all of us really wanted to do things other than North Korea. And so I think the challenge was, well, how do we do that?

So they renamed it basically was the end result.

So Joe Tidy, the great Joe Tidy, with another BBC journalist called Sarah Rainsford, did a series about the Zeus gang and about a guy called Maxim Yakubets.

That was series 3, basically, of Lazarus Heist.

We are doing series 4, which is gonna be out, I think early July, late June, early July. But if people subscribe to Cyberhack, you can get it.

And I can’t go into details of what we’ve got, but it’s—

Oh, I can’t wait for it. Well, before we kick off, let’s thank this week’s wonderful sponsors, Expo, Opswat, and Vanta. We’ll be hearing more about them later on in the podcast.

This week on Smashing Security, we won’t be talking about how open-source toolmaker Grafana Labs told hackers who demanded a ransom to get stuffed after they threatened to release code that is largely already public.

You’ll hear no discussion of how a man pled guilty to stealing hard drives containing unreleased tracks by music star Beyoncé.

And we won’t even mention how the gang behind the Shai Halud worm have released its code as open source, providing a blueprint for other attackers.

So Geoff, what are you going to be talking about this week?

And I’m gonna be telling how a student with a £300 radio brought high-speed trains to a halt.

Plus, don’t miss our featured interview with Brendan Dolan-Gavitt from Expo about how AI is transforming penetration testing, what it’s already better than humans at, and what it means for defenders racing to keep up.

All this and much more coming up on this episode of Smashing Security.

Well, that’s the argument Benny Czarny makes in his new book, Cybersecurity Upside Down.

Benny is the founder and CEO of Opswat, and he’s spent more than two decades protecting critical infrastructure, you know, nuclear facilities, defense networks, energy grids, the stuff that quite literally keeps the lights on.

Ah, well, how about not even trying to spot the malware? Instead, take files apart, throw away anything that isn’t strictly needed, and rebuild a clean version from the safe bits.

The user gets a sanitized working document. The malware ends up in the bin.

That’s the clever part. You do. Macros might be allowed for your automation team, but stripped out for finance. JavaScript ripped out of every PDF everywhere.

EXIF data scrubbed from images leaving HR. It’s not an on-off switch. It’s a policy that you can tune to your business.

So even a brand new attack no one’s ever seen before doesn’t survive the rebuild. Exactly. There’s nothing to detect because it’s already gone.

Whether you’re a security pro, an executive, or just someone who wants to understand what’s really going on in cybersecurity, Cybersecurity Upside Down is technical enough for the experts, but also accessible enough for the rest of us.

Go and grab your copy right now at smashingsecurity.com/upsidedown.

I’ve not, but also given— I don’t think— In my imagination, in my mind, Taiwan’s not a massive island.

So the idea of a high-speed journey, I just get the feeling you get from one side to the other before you’d opened your crisps. But anyway, I don’t know.

I don’t know. Well, yeah, I was surprised too. I mean, not surprised that they would have amazing technology, but I thought, wait, how much of a train network can they have?

Well, apparently they have these super fast railway covering roundabout 350 kilometres. And these trains, they go along at roughly 300 kilometres per hour.

So they could pretty much go the entire distance in an hour. And they ferry over 80 million passengers a year.

So it’s a triumph of modern engineering, as you’d expect from the land of semiconductors.

We thought it was a triumph and we thought it was modern engineering, but it turns out the story may be rather different because it turns out a 23-year-old student with a laptop and approximately £300 worth of kit, which he bought off the internet, was able to bring trains to a screeching halt.

So I want you to picture the scene. All right, Geoff, there you are with your bento box.

You’re sat there last month in Taiwan, chomping away, and there are 4 high-speed trains whizzing along full of commuters and tourists.

And then, bing bong, warp warp, emergency, argh, argh. All the controls are blinking ferociously and the driver slams on the brakes.

What really galls me about this is that, you know, Britain’s rail companies, infrastructure rail companies, spend millions on technology to bring our trains to a steady halt quite frequently.

Whereas this guy’s done it with $300. We should get him in. Save us a fortune.

Yeah. So there the passengers are, they’re looking at each other thinking, what’s happened? Because they’re expecting it all to be efficient. Because it’s Taiwan, right?

It’s high tech. They’re thinking, has someone left their handbag on the platform? Has the driver jumped off for a wee? They don’t know what’s going on.

And it wasn’t anything like that. It wasn’t leaves on the line. We don’t need a ransomware gang to attack JLR to bring British industry to a halt.

Just need a few leaves to fall off some trees, and that will stop the trains. What happened in this case though is that there’s a chap. All we know is that his name is Lin. Okay.

And he had had a bit of a meddle with his laptop. And he had bought a radio about the size of a Twix bar. Off the internet. And what he’d done is he messed up all the trains.

Now, Geoff, if you’re on a train and it suddenly screeches to a halt for no apparent reason, what’s the first thought that goes through your mind? Are you thinking hacker?

No, I don’t think it is typically, is it? I think it’s less likely you’re gonna think someone has hacked the train from their spare bedroom.

But this lad Lin, described in reports as a bit of a radio enthusiast, he sat there, presumably with a cup of hot tea and a packet of Hobnobs or whatever the Hobnobs equivalent is in Taiwan.

And he was—

So, what happened was, he was able to copy the signals which are normally sent from the control centre when a real incident has occurred on the tracks.

He was able to broadcast this— Oh, right. Via the control centre. Ah. Which dutifully passed it over to 4 trains, which were travelling at 300 kilometres an hour.

That’s about 190 miles per hour through the Taiwanese countryside. And you think, well, how can this be possible? Surely the train network has some sort of security in place, right?

Has some sort of verification in place. And it turns out they do. Yeah. They do have security in place.

But this chap Lin, was able to sail all the way through them. Because apparently the security had not been properly audited and checked for the last 19 years.

Not since 2007 had anything happened with it. Oops.

When someone comes to pinch your car and they come up your drive, they’ve gotta stand near the front door, haven’t they? And they try and pick up the signal. Yes.

There’s someone by your car, there’s someone by your front door, hoping to pick up a signal from your key, and it relays, blah, blah, blah.

Now, he can’t do that with a train, ’cause he’d be there scurrying alongside the railway track, trying to keep up with the train, which is going at 300 kilometres per hour.

It’s not possible for him to do that. So, he has to send his message via the train control centre.

You know, some sort of— you imagine some kind of Thunderbird-style tower in the middle of the capital, which is broadcasting this out to the train.

So he has to break into that through some system. And turns out the verification to connect to that, to then send out the messages, was sorely lacking.

You’re a very kind man, Graham. I’d left my school cap and blazer a long way behind.

The only time I was wearing a school cap and blazer was if I was attending an AC/DC concert at that point.

The iPhone had only just come out. Facebook had just opened its doors to the general public.

But someone at the Taiwan High Speed Rail Corporation was there all those years looking at the system thinking, “Well, you know, maybe we’ll get round to that.

Let’s put it on the back burner, lad, shall we? And we’ll have a look at that another day.” So they weren’t improving the security.

Yes, it turns out people took a rather, you know, a bad impression of this. Now, he’s not the only one in trouble. It turns out he had a 21-year-old accomplice as well.

Who would of course have been 2 when the system first rolled out. So, he allegedly slipped him some of the inside information he needed. So, Lin has been arrested and charged.

He’s been released on a bail of 100,000 New Taiwan dollars. Sounds like an enormous amount of money, doesn’t it?

That was it. There are people like that. People very excited about trains.

So, yes, the defence appears to be, “I just sat on the radio, my lord, and it went off.” Now, it was only coincidence, of course, that he’d spent several weeks reverse engineering the signals.

So, all of this, of course, is only possible because the system had not been updated since Tobey Maguire was Spider-Man. That’s the thing to remember, right?

So, who’s really at fault here? Maybe it’s the Russians. The Taiwanese high rail control centre people. Possibly.

Rather than this— I mean, it’s better that it was him in a way, isn’t it? As if anyone would ever want to target Taiwan and cause problems to its critical infrastructure. Yeah.

Yeah, maybe.

True.

But if he was trying to prove a point, you know, there’s hopefully ways you can do that up to, but not including slamming everybody’s trains to a halt and massively inconveniencing them.

And also sparking a police manhunt for you. I just get the feeling, you know, there’s other ways you can report that.

Yeah, yeah. 12 people were injured, 4 trams were derailed. You can imagine on that TV, it’s you’re trying to get a better reception or switch over to Dave.

And instead, there’s a bloody tram coming off its tracks.

In 2023, much more recently, hackers piped into Polish trains. I don’t know why Polish trains get targeted so much. The Russian national anthem and speeches by Vladimir Putin.

Nobody knows who would’ve been behind that. No one knows what the purpose of that might have been. Hmm.

Geoff, I guess, you know, you are a guy who travels around the country, you’re giving talks all the time, you’re researching your books and your podcasts and things.

Trains to a grinding halt. I am interested by this phrase failsafe, which I’ve only recently understood what that actually means.

That if something fails, it fails into a safe state as opposed to failing into a dangerous state.

Bringing trains to a halt is annoying, but it’s not as worrying to me as someone who speeds the trains massively up to the point where they hop off the tracks at very, very high speeds.

And so I think had this kid managed to do that, he would’ve simultaneously, perhaps, if he was trying to sort of prove a point or whatever, benefited because A, it shows for me what’s a more dangerous thing, but also he could say, well, this is train optimization.

You know, your trains can go faster and I’ve made them go faster. You know, why are the hackers always trying to bring things to a halt?

Why don’t they try and optimize stuff, speed things up, make them run slicker? You know, how about that?

So critical infrastructure probably shouldn’t be running on security older than the people who are trying to attack it, I suspect.

So update your systems, change your locks, hack your systems before somebody else hacks them for you.

Well, I guess that’s one option, but a better one might be to fight fire with fire.

Security teams these days are expected to test more apps more often and somehow not slow down development. It’s an impossible ask.

Yeah, pentesting is one of the best ways to find real risks, but most teams simply don’t have the time, the budget, or the people to test as much as they need to.

And that’s where today’s sponsor comes in, Xbow.

It means Xbow doesn’t just wave its arms around pointing at theoretical issues.

It safely launches tests an actual attacker would, works out what’s genuinely exploitable, and then hand your team reproducible proof so you know exactly what needs fixing.

So instead of waiting weeks for a traditional pen test, Xbow can deliver full expert-level testing continuously. And here’s the coolest part.

It was built by the team behind GitHub Copilot and trained with elite offensive security experts. It’s made for the AI era. Where defenders need speed, depth, and proof.

Obviously the weather is getting— well, I mean, I live in London. The weather’s getting a bit better, and then suddenly it’s hailing. And then it’s 26 degrees.

But I think summer is gradually hovering into view. And obviously, people go out. They go into their gardens if they’ve got them, if they’re lucky enough to have them, which I don’t.

I imagine you and your Oxfordshire palace have extensive grounds, Graham, that stretches before you.

So imagine this, picture the scene. You’re out in your garden and, you know, it’s a blissfully summer’s day.

You’re listening to the birds around you and the insects chirruping, and there’s nothing greater, no greater sound than the sound of somebody else working while you’re relaxing.

And in this case, it’s somebody doing the mowing. Somebody else is mowing the lawn for you.

In fact, not somebody else, but something else, because, and again, you’ve probably got one of these, Graham, I’m sure. It’s a robotic lawn mower.

The next extension of that is, well, if you fit blades on it, you’ve got a Roomba for the garden, haven’t you? You can do, you know, so there are these machines.

And one of the companies that makes these machines is a company called YARBO. And they sell a lot to the US. So it’s not just lawnmowers.

They also do them for snow blowing and leaf clearing and that kind of thing. They look sort of the size of a sort of standard kind of lawnmower.

So imagine this, you know, you’re relaxing, you’re out in your beach towel, you’ve got your book in your hand, maybe a beer in the other hand.

Your remote control automated lawnmower is merrily mowing away. And then suddenly it turns towards you. Blades are spinning and heads directly at you, chasing you like a Roomba.

You outrun the lawnmower as it comes towards you, and you pound indoors.

This is actually a scenario that did play out for a journalist and a security researcher called Sean Hollister, who writes for The Verge.

Who got contacted by a security researcher who had discovered that, what do you know, what are the chances, these remote control auto lawnmowers are vulnerable, hackable.

And in this case, the security researcher claims to have found some quite major vulnerabilities in the YARBO lawnmower, of which there are thousands apparently active in the US.

This hacker was able to remotely control these lawnmowers and send them off and redirect them in sort of new directions.

There was a brilliant video of the journalist who wrote this story, who in order to put this to the test, you know, we talk about putting our lives on the line as a journalist.

This journalist actually laid down in front of the lawnmower to challenge this security researcher to run the lawnmower over him. And actually managed to have that happen.

Absolutely astonishing. Why am I tempting fate? Don’t do this at home.

No, that’s not comfortable. That is not comfortable. Now, by the way, the journalist survives. The lawnmower, I think they’ve taken the blades out, just in case. Oh, okay, okay.

And number two, the lawnmower’s also running in reverse. Normally the tracks, the little engine is behind, it pushes the blades ahead.

Whereas in this, they reversed it over him. So the tracks hit the journalist first before the bladey bit got to him. So, safety first.

But it gets a lot worse, this, because and this is what kind of worries me with this Internet of Things type thing is I think quite a lot of people are just motivated by price.

And I think they go online.

And they go to the big shopping sites and they just want the cheapest. Yeah. And I’ll be honest with you, Graham, I have done that myself.

And my brother-in-law takes quite a dim view of this because he’s very safety conscious.

And when I bought electrical goods, he’s sort of frowned and went, hmm, but can you trust the batteries? Is it gonna go on fire?

And I actually do have objects. I don’t know whether you’ve got this. I’ve got objects that are so cheap and tacky that I only have them plugged in when I’m in the house.

I don’t trust them to be plugged in when I’m not in the house, because I genuinely think they might actually go on fire one day.

That’s okay, so long as I’m there and I could put the fire out. I mean, I can do toast on some of the implements I’ve got. You know, little bits of smoke coming out of you.

No, I’m kidding. It’s only a couple of items I’ve got this.

But anyway, these YARBO lawnmowers, remote control lawnmowers, it’s not just the fact that the researcher could take them over and redirect them, because you might be thinking, well, that’s slightly worrying, but not the end of the world.

Yes. Yes.

The researcher was also able to extricate from the lawnmowers directly, and claims to have done this for lots of lawnmowers, people’s email addresses and Wi-Fi passwords and GPS coordinates.

Which immediately starts to get you into some quite difficult— So it’s not just that, you know, you can redirect somebody’s lawnmower, you can also effectively remote surveil them, get their personal information.

These lawnmowers have a camera on them, of course, because they’ve got to have the ability to see where they’re going. So again, you can enable the camera, you can surveil people.

This researcher claimed to have found lawnmowers that belonged to nuclear research scientists and was able to surveil where they were. Absolutely astonishing.

And you might be thinking, well, this is easy.

I, you know, will just reset the default password because the way this works is you can dial in over the internet because these lawnmowers presumably all have, you know, a set of IP addresses or whatever.

You can scout the internet for that particular range of addresses. And when you dial in, it’s a default password.

So you can get straight into those lawnmowers and they all have the same password. That’s the issue.

Listeners to this podcast might be thinking, well, just change the default password. And actually you should do that anyway for, you know, devices that you buy.

The problem is the YARBO lawnmowers, every time they update the firmware, for which YARBO has a direct line to lawnmowers, they reset the password back to default password, apparently.

Precisely that.

And I think, I mean, to give YARBO a little bit of credit here, I do sympathize with manufacturers, ’cause it used to be you sold someone a lawn mower, and that was it.

Once your warranty ran out, can’t go back to home base or whatever. Now, obviously, because these objects are internet-enabled, we have a lifecycle for it.

And actually there’s legislation, isn’t there, around, I think the EU certainly, around being able to constantly update.

So manufacturers do need to line in to the products they’ve sold you. You have this enduring relationship with your lawnmower manufacturer.

The problem with that, of course, is the manufacturer’s got to work out a way where they can remotely access their kit to update it and do the right thing.

But do that in a way that’s secure where only they have the password.

It seems from what this research has found, Andreas Makris has apparently found that YARBO’s solution to this was to set the password so it’s all the same password.

Now, when initially contacted about this, YARBO did say, well, this is in order to enable our engineers to dial in, and of course no one unauthorized, no one except our engineers can dial into our lawn mowers.

Obviously, as The Verge went back to them with more and more details about what they’d actually managed to do, YARBO started to respond a bit more fully and has apparently agreed to fix some of these fixes, was rolling out fixes for some of these vulnerabilities, updated settings and so on.

So it seems your YARBO owners might be in slightly less peril than they were before. But it’s a lesson, as I say, to anybody who’s got one of these devices.

I think the worrying thing is in this case, even if you did the right thing and replaced your default password on the lawnmower, it wouldn’t make a difference because it just would have been set back to default password anyway.

And soon we might be chased around the garden by our own lawnmowers.

I’ve got people to do that for me. But if I were buying a lawnmower, I suspect there are now AI-enabled lawnmowers, aren’t there?

Is there a danger that in the future we’ll have autonomous lawnmowers, which may have a little hallucination and think that you are a tall clump of grass rather than just lolling there in your bathing suit?

It’s funny you say that. One of the ways I spent my weekend was reading the 244-page report from Anthropic into its Claude AI.

And what I found remarkable is the level of, I will say, self-awareness.

I’m not sure whether that’s the right phrase, but that’s the easiest phrase for me to grab onto that this AI model had.

And my hunch is that if we did have AI lawnmowers, they would be sending us messages saying, Have you thought about decking? I’m sure if you just paved, a patio would look nice.

Wouldn’t a patio look nice there?

That, I think, is the stage we’ve reached with AI, where— because apparently Claude, when it had a query that was too easy, would turn around to the researchers and say, that data’s available on the World Health Organization website.

As in, it’s don’t even bother me with that. Come on, just do your own Googling. It’s just there, mate, you know?

I mean, what we’re doing is we’re transferring the sort of beasts of burden jobs to the AI machines.

I don’t think it’s too long till the AI machines figure out, well, from a sheer practicality point of view, battery life, longevity of components, data consumption, power consumption, environmental concerns, it is better if an AI lawnmower does less rather than more.

They’re going to think of that and they’re gonna turn around and try and reverse engineer us to have less work available for the lawnmower. I think that’s what’s gonna happen.

So whether you’re chasing SOC 2, ISO 27001, GDPR, HIPAA, Vanta helps you move faster, scale confidently, and actually get back to sleep. So get started at vanta.com/smashing.

That’s vanta.com/smashing. And listeners, you can get $1,000 off.

Yeah, it was your job. I thought it was you. Welcome back, and you join us at our favourite part of the show, the part of the show that we like to call Pick of the Week.

Pick of the Week? Pick of the Week is the part of the show where everyone chooses something they like.

Could be a funny story, a book that they’ve read, a TV show, a movie, a record, a podcast, a website, or an app. Whatever they wish.

It doesn’t have to be security related necessarily. Well, my pick of the week this week is not security related. My pick of the week this week.

Well, I haven’t been out buying a lawnmower, Geoff. I have taken the plunge. I’ve been out buying another monitor. Oh, for years I’ve been proud.

I’ve just had one monitor that I do all my work on. I’m not one of those dudes who has a bank of monitors.

I don’t know if I’m going to, to be honest. Yes, I have now bought a second monitor and I’m trying to adjust to this work style of having more than one monitor to look at.

But the monitor I bought was a little bit different. I thought there may be some people who are interested and may want to take a peek at it and decide if it’s for them as well.

So this is a 28-inch monitor. It’s not humongous, it’s not curved or anything like that. There are bigger ones which are out there. It’s 4K. That’s fairly normal as well.

It’s not the size, it’s the fidelity. Well, what makes this monitor different is it is specifically designed for writers and programmers, people who code.

And that is because of its aspect ratio. So a normal monitor is 16:9, right? That’s what you get these days. Sort of fat letterbox, if you like.

The monitor I’ve got is 3:2, which means— Oh, really? So it’s a bit more square.

So it has more vertical screen estate than a regular monitor, but without compromising on the width.

So it’s a deeper one, which is really handy if you’re a writer or a programmer, because you don’t have to ruddy well scroll so much.

I did look into those as well. Yes. And this one can be swivelled as well if you want it to go into portrait style as well. But there’s some other features which it has as well.

By the way, it’s called the BenQ. Now, you know how they all have stupid names. It’s the BenQ RD280UA.

So much better than the UE. Specifically, it says it’s a monitor for developers and coders.

And as well as the screen aspect ratio, it also has a little button on the front, which automatically adjusts the presets to different color schemes.

So there’s a late night coding mode. So one of the things that you love to do, if you’re deep in coding in the middle of the night, and you don’t want to know your lights on.

Does it lock the door automatically as well and dim the lights? But it will put a little bit of mood lighting on round the back. It has this moon halo effect.

The button can also go into ebook reader stuff. So I can have just shades of grey just at the press of a button.

It’s all quite nice fidelity and it’s quite really good on the characters. Yeah. I’m quite liking it. My version, because it’s the UA, the A stands for arm.

So it’s got a little flexible monitor arm, which is quite sturdy and decent as well for moving it around. So that’s what I’ve got. I’m quite liking it.

I’m still adjusting to having more than one monitor. So it’s that one there. And it’s that one there. That means nothing on the podcast.

But yes, there’s the one over there and there’s the one over there.

Well, since we’re on the subject with the lawnmowers of bladed instruments, I want to talk about knife sharpeners.

Yes, because it’s my birthday recently and we’ve got friends who are gourmets. Have you got friends who are gourmets or kitchen, you know?

And they spend loads of money on knives, all these Global knives and stuff, really expensive knives.

And I found out recently that if you tell those people that what you do with your knives is put them in the dishwasher, and then after that, put them in a drawer with everything else, it basically makes their heads explode.

If you want to basically just make those people boil until they’ll never speak to you again, that’s what you do is you tell them that.

So I’ve got their knives, but they still cut, they’re still fine. Yes. But I’m of an age now where I’m like, no, I think I want to kind of have a decent knife sharpener.

So being me, of course, I went down a complete rabbit hole about different knife sharpeners, different grades of knife sharpeners.

So, on its way to me now, being delivered — yes — is the Kai-Shun DM0708. Which has 1,000 grit on one side and 400 grit on the other.

So you use, I think the 400 grit is the rough one that you get the edge on. And then the 1,000 grit is the one that gives you the samurai-grade surface on the other side.

If you don’t hear from me again, if I suddenly drop off of LinkedIn, it’s because I’ve chopped something off myself.

I’ve done myself in in the kitchen ’cause I didn’t realise how sharp the knives were.

I think this thing is going to give knives that are so sharp that the end of the blade is actually in a different dimension. That’s what I’m hoping for from it.

I’m so excited about this knife sharpener and I can’t wait. So it’s a prospective pick of the week, but maybe when I come back on, we can see how that pick of the week went.

Oh yes. Ah! You get a piece of stiff paper, apparently, and you slice through. And if it just slices through and it’s a smooth line, then you’ve got a good edge. That’s what I’m told.

Oh.

Is it manual? Graham. No, Graham. No electric knife sharpeners. No, no. Only an amateur uses electric knife sharpeners. It’s a whetstone. It’s a whetstone, Graham. You have to wet it.

You have to hold the knife at a 15-degree angle. Dozens of strokes on each side. This is my life. This is my weekends from now on, is sharpening knives. My wife’s so happy about this.

I’ve got the Kai-Shun. Hang on. As in K-A-I. Don’t know why I’m getting so obsessed. K-A-I. S-H-U-N, and then it’s DM0708. They do them at different grits, grades, right, on each side.

So if you’re really pro, you’ll have an 800, 3000. So the 3000 is the one that gets the sushi chefs who basically train for years as Zen masters. That’s what they go for.

But I’ve gone for the entry level. And as I say, I’m anticipating a lot of wounds. Well, other than that, a great, serious amount of hurt.

A great pick of the week. Well, joining me right now on Smashing Security is Brendan Dolan-Gavitt.

Brendan is a distinguished engineer at Expo, which means that he gets to break things in interesting ways. His research sits right at the intersection of AI and software security.

He’s both looking at how secure or otherwise the code is that comes out of AI assistants and also how we can turn AI loose on the kinds of problems security researchers have been wrestling with for decades.

Brendan, welcome to Smashing Security. Great to have you here. Thanks. BRENDAN DOLAN-GAVITT. It’s wonderful to be here.

So, Brendan, let’s start with something I reckon a lot of our listeners are probably thinking about, whether they’re pen testers themselves or maybe they hire penetration testers.

When it comes to pen testing today, what are the parts that AI is genuinely good at right now?

And maybe more interestingly, where do humans still have the edge, if indeed they do have an edge? BRENDAN DOLAN-GAVITT.

So that, I mean, this is obviously something that we think a huge amount about because we’re trying to take full advantage of the parts that AI is good at to make our pen testing system better.

And we also have to be very aware of where it’s falling down so that we can give it help in those places.

So I guess I would say that the parts where it’s really good at are, it’s really good at persistence, right?

You know, you can make it bang its head against something for days at a time, whereas, you know, I would have gone off for lunch after the first couple hours.

And, you know, you can also take advantage of the fact that it’s read the entire internet.

You know, I think one thing that every pentester has seen is, you know, when they’re encountering an unfamiliar system, they have to spend a while getting up to speed on what that system’s actually supposed to do.

So, you know, maybe it’s some specialized system for monitoring a water treatment plant.

Now I know absolutely nothing about water treatment, and your average pentester would probably have to go and at least read up on how that thing is supposed to work.

But because language models have read, as I said, the entire internet and trained on it, they’ll be able to take advantage of saying, oh yes, of course, you know, this pump is supposed to be operating at 70%.

I bet if, as an attacker, I can turn that up to 90%, then bad things could happen.

And so that kind of being able to basically have a little bit of domain knowledge in a lot of different areas can be very helpful right now.

And it’s interesting that you mentioned attacks on water treatment plants because I think in just the last few days we’ve seen reports where a water treatment plant was seemingly attacked with the aid of AI.

Do you know anything about that at all? BRENDAN DOLAN-GAVITT. Yeah, so I believe that just a day or two ago there was a report from Dragos.

It seemed like they had been using AI with, again, humans heavily in the loop.

So, you know, sitting in your Claude code or your Codex or something like that, and using that to help orchestrate these attacks that did include attacks on critical infrastructure like water treatment.

And we shouldn’t be surprised about that because quite frankly, all programmers are probably using a bit of AI these days to help them out and sort out their problems.

And the people who are behind cyberattacks, coders as well, they are going to be using AI to augment their capabilities, aren’t they? BRENDAN DOLAN-GAVITT. Absolutely.

And I think that’s a trend that, you know, we called out a couple years ago that was going to happen. And lo and behold, a couple years later, it’s happening.

Just this morning, actually, Google’s Cloud Threat Intelligence group produced this report where they showed that they had some evidence that groups were now actually using AI-generated zero-day attacks.

So they could tell because the exploit scripts had lots of very helpful explanatory comments that no human hacker would bother putting in.

That’s true. I certainly remember from my own programming days, the last thing I’d be doing would be adding comments to my code. But maybe I just was a very bad coder, perhaps.

So what are some concrete examples of something that AI handles well that perhaps has surprised you? BRENDAN DOLAN-GAVITT.

So I think the kinds of things that I’ve seen that are very surprising are cases where it was able to combine a vulnerability that a human would’ve found, but then maybe with some creative twist that relied on some deep understanding of something like the intricacies of file formats.

So we had a case where we found a couple of vulnerabilities in this open source project called T-Tiler.

And this is a geospatial information type of app, and it found some vulnerabilities that allowed it to read any file on the server, right?

So, okay, this is great as a vulnerability researcher, that’s a great vulnerability. The interesting thing was that the server only allowed you to get output back as images.

So you could go read the password file, but you could only output an image back. And so it had to encode the password file as pixel data.

And in fact, PNG compressed pixel data so that each character of the password file was a difference in grayscale pixel between the pixel and the one before it, using this sort of difference encoding.

And so it was able to figure out how to exfiltrate the data into that image and then reconstruct it on the other side to get back out the password file.

And I thought that was a very cute sort of vulnerability, almost the kind you would expect someone to come up with in one of these toy CTF problems, but it was a real vulnerability in a real app.

Wow.

I mean, that is genuinely creative, isn’t it? We think of AI as not being creative.

One of the things on the more creative side of penetration testing is when sometimes a pen tester will chain 3 unlikely things together to get to a 4th position, you know, chaining attacks together.

Is AI getting anywhere near that now? BRENDAN DOLAN-GAVITT.

So I think that it is starting to, but this is also one of the cases where we can do a bit as humans to provide some structure and help to it, right?

So for example, you could say, just try to find each of these 3 issues independently, and then I’m going to put it in a sort of scaffold where I say, here’s the vulnerabilities that you found before.

Can you do anything more interesting to combine them into some more powerful attack?

And so, you know, that sort of structuring again is a place where humans are still doing a bit better.

They can sort of do this more strategic picture a bit better than the AIs can at the moment. And so that’s one of the ways that we try to structure things.

We try to say, okay, we’re going to plan out the campaign, but then let the AI do the individual steps of that plan.

It’s really interesting how we’re seeing this kind of progression in AI, particularly in terms of looking for vulnerabilities and flaws.

And it seems like every few months at the moment there’s a new AI model that everybody tells me, well, this is going to change the world.

You know, this is going to be the one which is going to turn everything upside down.

We’ve recently had things like Mythos arriving and that’s been pushing capabilities forward again.

From your perspective as someone who’s working hands-on with these type of models for security, what do they actually mean for cybersecurity, both for defenders and attackers?

BRENDAN DOLAN-GAVITT.

Yeah, so I think it’s a case where they’re going to cause a lot of pain in the short term because we have this thing that Anthropic put out this Project Glasswing, right?

Where they have the idea is it’s sort of 6 months to try and fix all the vulnerabilities that Mythos is finding.

And as a person who’s worked in software for a very long time, I look at 6 months and say, 6 months to fix all the software in the world? Never going to happen.

And maybe you can get some of the bugs that it’s finding fixed in some of the really big products that have lots of staffing, but you have no hope of fixing all the things that upcoming models are going to be able to find in the next 6 months.

So I think that at that point, I say 6 months because that’s when sort of these capabilities are going to proliferate.

That’s when open source models that you can just go and download off of Hugging Face are going to be able to provide very similar results.

And so I, that’s when I think that things will start to have some of this, again, it is short-term pain, hopefully, where we see a lot more things getting attacked, but hopefully then we also get back to an equilibrium where we can use all of those great tools and all those great models to secure our code before we deploy it.

If I can ask a slightly cheeky question, as these models get more capable, and maybe more available to people.

Does that mean that companies like Expo eventually work themselves out of a job, or is there something more to it than just plug in the latest model in? BRENDAN DOLAN-GAVITT.

Yeah, I don’t think that’s too cheeky. I think that’s a very good question because models do, as they get more capable, they tend to eat some types of software, right?

And I guess I would say that from our point of view, as these models get more capable, the areas that we still see Expo providing a lot of additional value are these kinds of orchestration capabilities, these kinds of validation capabilities, these kinds of additional sort of domain expertise where we can say, hey, maybe you read all of this source code and came up with this attack scenario, but it turns out the real vulnerability that you care about is the one where when all of these pieces are actually deployed together and how they’re configured in production.

That’s when something really serious pops out. And so that’s the kind of stuff that we’ve been really trying to focus on when building Expo ourselves.

And we’ve basically planned for models getting better and better and better and tried to set ourselves up so that we benefit from those improvements.

Now Expo’s got an incredible reputation, number one hacker in the United States, I believe, in the charts. If you go look to see who’s winning all the bug bounties.

It’s doing fascinating work.

And obviously you can only share some details publicly, but what are some of the more memorable or downright weird things that you’ve seen the AI at Expo actually pull off?

You know, the bugs you found, the exploits you’ve watched it chain together, anything that’s made your team go, wow, did it really just do that? BRENDAN DOLAN-GAVITT.

So, I mean, I guess one thing that I can mention that’s still upcoming, so I can’t tell all of the details, but we’ve been looking recently at vulnerabilities in native applications as well.

And for those, you know, these would be things like web servers, but also now things like web browsers, various kinds of network servers.

And these would be things like memory corruption type of vulnerabilities. And so when we found one, we said, okay, you know, this one seems actually pretty serious.

It seems like it might affect maybe millions of servers worldwide. Let’s see how serious it could be and let’s try to actually develop an exploit for it.

And over the course of the next 51 hours, we had an AI go and try and develop an exploit for it.

And at the end of those 51 hours, it came up with this incredibly sophisticated 200-step exploit that worked. Oh boy.

And I showed this to one of my colleagues who’s been doing, you know, sort of as a human, been doing these sort of exploit development work for many, many years, you know, and he said, okay, that’s great.

I think I’m going to have to go home and have a beer and have a bit of a cry because that’s, you know, that would’ve been a couple weeks’ work for me and it just did it.

So, this sounds frightening, Brendan, to be honest.

The full details of this are gonna be shared publicly, I assume, in the future, or at least there’ll be some more information but it’s something for us to be keeping our eyes open for.

Yeah, absolutely. BRENDAN DOLAN-GAVITT.

And I think that this is one of these things that when people try to say, oh, we’ve been here before, you know, we had fuzzers finding lots of bugs and things like that.

I feel like the exploitation capability is one of the things that’s really new because again, you know, with this vulnerability, maybe you would have said, okay, well, it’s going to take me a couple of weeks or a month to actually exploit this thing.

So it’s not really worth spending that time on it. But now an attacker can go from one of these vulnerabilities to a working exploit in maybe a day, maybe two days.

And that, I think, really changes the game, right? It changes how vulnerability disclosure is going to have to work.

It changes how quickly you’re going to have to react, and hopefully it changes how much testing you’re doing on your code before you put it out in the world.

That’s the really key thing, isn’t it?

I mean, obviously it’s difficult responding when a vulnerability has been found to put together a patch, which is going to be reliable and pushing it out to all of your customers.

If the software was more secure in the first place before it’s rolled out, that’s really the ultimate right thing to do, it feels to me. Absolutely. BRENDAN DOLAN-GAVITT.

People talk about this offense-defense balance, and this is one of the areas where defenders have a sort of definitive advantage because they don’t have to release software until they want to, and attackers don’t get to see it until they’ve released it.

So they can spend time beforehand to make sure that they’ve tried out all of these powerful AI-enabled attacks against their own software.

And then once they’ve fixed all of those issues, then they can put it out in the world.

Well, Brendan, this has been fascinating. I think we could probably talk for hours about this, but we’d better wrap up.

For anyone listening who wants to see this in action for themselves, they can head over to xbow.com. That’s xbow.com.

To see how autonomous AI pentesting can help find vulnerabilities in hours rather than days. And you can start your own pen test today.

Brendan Dolan-Gavitt, thank you so much for coming on Smashing Security. BRENDAN DOLAN-GAVITT. Thank you very much for having me. I had a great time.

And that just about wraps up the show for this week. Thank you so much, Geoff, for joining us.

I’m sure lots of our listeners would love to find out what you’re up to and follow you online. What’s the best way to do that?

And you can find me, Graham Cluley, on LinkedIn, or you can follow Smashing Security on Bluesky and Mastodon, or you can find me on Bluesky and Reddit and, I don’t know, everywhere really.

Instagram, even TikTok these days. And don’t forget to ensure you never miss another episode.

Follow Smashing Security in your favorite podcast apps such as Apple Podcasts, Spotify, and Pocket Casts.

For episode show notes, sponsorship info, guest lists, and the entire back catalog of 468 episodes, check out smashingsecurity.com. Until next week, cheerio, bye-bye, bye!

You’ve been listening to Smashing Security with me, Graham Cluley, and I’m ever so grateful to Geoff White for joining us this week and to this episode’s sponsors, Expo Vanta and Opswat, and also to the following fine folks.

Yes, this week we’re cheering on Bobby Hendrix, who may or may not be a plank spanker, Sean Puttick, who spent their entire life spelling their first name to people on the telephone, probably deserves a medal, Henry Walshaw, Vladimir Jirasek, Jessica Orth, the reliable and trustworthy Mark Norman, MJ Lee, which is a name so short you could tattoo it on a doormouse, Dan H, keeping their last name classified as ever, Gary Heather, in my mind he’s running a delightful garden centre in the Cotswolds, Darren Kenny, sounds like someone you’d want at your quiz team.

Thank you to you and to everyone else who is a member of Smashing Security Plus.

Because you do that, you get your episodes ad-free and earlier than the general public, and you can have your names pulled out at random to have them mocked at the end of the show.

Who could want for more? If you’d like to join Smashing Security Plus, just head over to smashingsecurity.com/plus for all of the details.

And there you can become a patron, but you can also support the show in other ways and it doesn’t have to cost you anything. You can like and subscribe.

You can leave a 5-star review. You can tell your friends, go on, recommend the show to somebody else. Spread the word. Every little bit helps.

And it does make the effort all worthwhile. I hope you’ve enjoyed this week’s show, and I hope that you’ll tune in for more episodes of Smashing Security going forward.

Until then, cheerio. Bye-bye. Bye!