Episode Cover Image

15. DDoS Attacks, Botnets, & Security Overwhelm.

Our guest's LinkedIn profile

Dissecting Popular IT Nerds
Dissecting Popular IT Nerds
15. DDoS Attacks, Botnets, & Security Overwhelm.
Loading
/

DDoS Attacks, Botnets, & Security Overwhelm discussed with one of the security industry’s secret weapons Mike Benjamin.

————

LINKEDIN

Telecom Radio One

Your solutions provider for comprehensive, end-to-end telecommunications services.  From the carrier, cloud, and IP infrastructure services to contract negotiation, issue resolution, and every matter in between — we do it all as we are the premier master agency for connectivity, cloud, and cloud enablement.

Disclaimer: The views, thoughts, and opinions expressed by guests on this podcast are solely their own and do not necessarily reflect the views or positions of their employers, affiliates, organizations, or any other entities. The content provided is for informational purposes only and should not be considered professional advice. The podcast hosts and producers are not responsible for any actions taken based on the discussions in the episodes. We encourage listeners to consult with a professional or conduct their own research before making any decisions based on the content of this podcast

DDoS Attacks, Botnets, & Security Overwhelm discussed

3 Key Takeaways

Episode Show Notes

Transcript

Speaker 0 | 00:00.544

Welcome everyone to Telecom Radio 1 and today we have a very, very complicated subject. I don’t even know where to begin with this. It’s called security. We’re not talking about people in hoodies hiding behind computer screens or anything. We’re actually going to delve in with someone that is very knowledgeable on this subject. Today we have Mike Benjamin on from CenturyLink. Mike, welcome to the show. Thank you very much.

Speaker 1 | 00:25.824

Yeah, thanks for having me, Phil. Looking forward to talking about security. Yeah.

Speaker 0 | 00:29.427

Now- You have been described, you know, I have my own description, but you’ve been described as the man that has been hiding behind the iron curtain for years. I like to describe you as the most popular unknown man in security. And that’s for a very good reason, because we don’t really want the very knowledgeable people to be known and, you know, where they are and what they’re doing, because we don’t want them to come after you. Actually, I don’t really know what that is, because… security is not supposed to be all fear-mongering. We’re actually supposed to have people that know what they’re doing. But give me a little bit of background on yourself, maybe a great story. Just tell me a little bit about what you’ve been doing for the last 10 years.

Speaker 1 | 01:08.780

Yeah, absolutely. So I lead a team here at CenturyLink called the Threat Research Labs Group. And you’re right, as we were building this team, we made a real focused decision to not name individuals from the group. We did that for a variety of reasons. One, as any group knows, You want to aim for group success, not individual success. So that was an easy one just for morale of our team. That’s the sort of less interesting side as opposed to the audience. But realistically, when we were out there, we knew that one of the goals for our group was to break criminal infrastructure, to take down botnets, to impact their ability to actually operate. And we knew just inevitably we’d anger some people. So why publicize the individuals from the group? You know, that didn’t make a lot of sense. to us. And so as we’ve progressed, we’ve done a lot of work with other security companies. We’ve done a lot of work with the industry and the infrastructure providers of the internet to literally clean up what’s going on that’s bad on the internet. And we believe we’ve made a material impact. And so, you know, you may have mentioned to me, you know, hiding, so to speak. You know, realistically, we had a reason to now talk more publicly about it. And it’s, we want to get more people involved. We want to get more of the internet, more of the… operators of that infrastructure to help us impact the infrastructure, clean up the internet, and collaborate on what, like you described, is a massive problem that a lot of people don’t know where to even start with. And so we’ve been working for, we’ll call it the last three years, really specifically on one particular problem I’ll describe to you, which is the DDoS botnet space. And so there was a lot of press a couple years ago with the Mirai malware family causing attacks that were… impacting some pretty big infrastructure and big websites. And we were fortunate enough to be in a position to have a lot of visibility of what was going on and have had a lot of impact on minimizing it since that time period. And so we were able to pull together some working groups. We were able to join some other working groups of people that had a like-minded ability to clean up the Internet, had time in their day. And in a lot of cases, I’ll tell you, these are people who do security work during the day, help their company operate. and then they go home and they have a real interest in doing this with us off hours. So some of them have brought that back to work. Some of them just continue to do it off hours. And so over the time period that we’ve been doing it, we’ve watched hundreds and hundreds of instances of these botnets stand up, and we’ve worked to take them down. We actually had a funny story where one of them actually registered a domain, one of the operators that contained the F word and then our company name. as the domain they put their command and control, their botnet on. So we’d made them so mad that they decided to laugh down at us a little and, you know, essentially tell us to step off in regards to that. So it was pretty satisfying for the group to see, you know, we knew we were making an impact on their day as we saw that register.

Speaker 0 | 04:12.036

Nice. Were you able to take down that botnet?

Speaker 1 | 04:15.078

We were. We were. I actually tried to come up, you know, I’m not a lawyer, but I tried to come up with… some legal reason why we can actually take ownership of the domain and redirect it to our security website. I couldn’t convince anybody of that. But yeah, we were able to take it down just like we do the others.

Speaker 0 | 04:30.931

Nice, nice. So maybe just for the layman listening and like myself, and just for other people that might be listening that don’t know exactly what a DDoS attack is or what a botnet is, maybe just give us just the general, you know, give us the general overview for anyone that might be listening that has no clue what that is.

Speaker 1 | 04:49.992

Yeah, absolutely. So we can imagine criminals want to do bad stuff to people, but they don’t want to use their own home computer. That’s a little too easy to find. It’s a little too easy to track, a little too easy to understand. And so they want to obtain other infrastructure to do it from. And so they use a lot of different ways. In some cases, they just even go out and buy virtual machines. They may even pay money. That money may not be so legally obtained in the first place, but in other cases, they break into computers. They also try to infect people with malware where they can control the computer. And depending on their scope and goals, They want as much infrastructure as possible in order to launch these attacks. And in the case of botnets, what we see is what they’ve done is they’ve said, you know, whatever they’re trying to carry out, we see a botnet deployed for spam. We see it for denial of service attacks. We see it for data exfiltration. They want a lot of computers in order to make the maximum impact on what they’re trying to accomplish. So DDoS specifically to your question. is at its core, it’s a denial of service. And you think what those words mean is it overwhelms something, so it can’t do its core purpose. So in the most simple terms, if somebody has a gigabit Ethernet loop to their office, if the bad guys can send over a gigabit of traffic to that connection, obviously the connection can’t do anything real with that capacity. In other cases, it could be a web server. A web server might be configured, you know, if somebody stands up… taxi on their Linux machine and says, hey, I want to serve up to a thousand connections a second. Well, in that case, bad guys might just send a thousand and one connections. There you go. The web server can’t do what it’s supposed to do at its core. And so that’s what the denial of service aims to do. However, as internet infrastructure has improved, the authentication services have matured, it’s become harder and harder to actually carry out what people aim to do on the criminal side. And so they started to distribute the attack. So rather than buying one VM and launching an attack or breaking into two or three servers and launching it, in some cases they want thousands, tens of thousands, hundreds of thousands. We’ve seen over a million in some instances computers on the Internet in which to launch these attacks. And if you can imagine what a million cable modems could generate on traffic, it’s pretty substantive. And so people’s homes even end up being participants in this, as we see in the IoT DDoS botnet space.

Speaker 0 | 07:22.911

Especially if they’ve got a 400 meg connection that they don’t need at their house for two people. But it’s interesting. I was reading your security report. And you guys had, from 2017, you said that there was 1.8 million. You had the top five bot hosting countries. And you had 1.8 million as a number next to the United States. And then you had China at 454,000. Now, that was a daily average. So are you saying that 1.8 million devices in the United States are being taken over and or used actively on a daily basis?

Speaker 1 | 07:59.532

Yeah, that’s what those numbers represent. And so what we’re utilizing is the intelligence data we’ve been able to collect from a variety of sources, from watching the botnets operate, tearing apart their malware, allowing them to attack us inside of Honeypots and other things that are intentionally built to be attacked. And we put together a reputation system with an understanding of where threats are in the Internet. And so the understanding of a malware family and what the main control infrastructure of a malware family is, is known to a lot of threat intelligence companies. Where we’ve taken that data a step further, and that’s where those numbers come from that you’re quoting, is we can, from a network perspective, actually look and see what is being controlled by those hosts. So if somebody installs a piece of malware, If you were to just look at the network connectivity, you could say, hey, it talks to that server. That server must be bad. But you don’t really have an understanding of is that server controlling 10 computers, 100,000 computers? What are they doing? How often are they hacking? And that’s the kind of information we aim to glean from that larger data set.

Speaker 0 | 09:07.216

Awesome. How much knowledge would you say the typical IT director, maybe CTO or CIO, how much? How much typical security knowledge does an IT director have, would you say, that you guys see just in general, and how much help do they need in the security realm?

Speaker 1 | 09:29.549

Well, the easy answer is it varies pretty widely, but I’ll try and quantify that a little bit more. So at the largest of enterprises…

Speaker 0 | 09:39.074

It’s very vague. I get that it’s a very vague question. Maybe I should be more specific around… around the botnet DDoS attacks. Obviously, typically, your IT director is like, hey, I can manage my firewall. I manage my network. They’ve got their… their general strengths let’s just put it this way what would you say is a let me ask it this way what is a typical weakness or what is a a typical uh area of of knowledge or expertise where people are lacking so that’s that’s a really good question the

Speaker 1 | 10:15.617

area that i see uh your you know sort of average business having the most common weakness across them is knowledge of their own environment so if you look at the security industry there’s a lot of folks that are from a consumer of that technology, right? They’re out and they’re trying to buy a better appliance. They’re trying to buy a better piece of software. And what they’re trying to augment is really their control. So your controls in the security world are the things that block stuff or minimize the ability to do stuff. And so antivirus is a control. A firewall is a control. Everything that sits in that sort of technology vertical largely is a control. So we see a lot of companies out there investing a lot of money in buying controls, and that’s good, right? That means they can block more stuff. That means they have more ability and more places to inspect and look for the good versus the bad. The problem we see, however, is they don’t know what they’re protecting. So if you take a step back and you think about your average company, how good is their actual inventory? Do they actually know where all the servers are, all the applications? And as we’ve seen recently, as we’ve seen libraries, become exploited? Do they understand what they’re writing software against? What are they linking into their applications that they are developing, whether it be a public or internal? They don’t know what they have. And so you can spend all the money in the world and buy every control on earth. If you don’t put it in the right places and turn on the right mitigations and understand your inventory adequately, they’re not going to do what they need to do. So that’s the number one thing we see. The second, though, is security. And this is a… a sad thing to say as an industry, but ultimately you can’t stop everything. No security solution on earth is a hundred percent effective. And so with that knowledge in mind and with that realization, people need to also be looking at how do they log what’s going on and how do they watch and then how do they react? And so that’s where that intelligence part feeds in is if people have an understanding of where their stuff is, they’ve put the right controls in. and then they watch what’s happening to it. They can then react and minimize when that inevitable break-in does happen because they can isolate to one server, keep it to five minutes. Therefore, the actor’s not been able to carry out on whatever goals they have. And so recommendations to a business looking at security is if they’re not asking all three of those questions, then they are not securing their company adequately. So where’s their stuff? How are they deploying controls to stop it? And then how are they monitoring at the end of the day?

Speaker 0 | 12:54.924

I had a follow-up question here. it’s kind of really off in left field, but I find that a lot of people, when they don’t have knowledge on a subject, they typically go to the internet and start searching for it. So as a complete side note, let’s just say you’re in charge of your entire company, your IT director, and you don’t have an answer to something. I find a lot of people going to various IT forums, okay? I just want, I just… I just want to ask you, what do you think of Reddit, maybe Spiceworks, some other IT forums? Should people trust these groups all the time? Should they be going and getting their information there? Just curious, complete side note, side question.

Speaker 1 | 13:40.140

It is a great question. Interestingly enough, people actually ask me, how do I keep abreast of the security market, the news, the things going on? And I tell them it’s a combination of this. massive rats nest of RSS feeds I’ve compiled over the years and Twitter. And so I do also subscribe to the NetSec subreddit and I do read that. But realistically, no one news source could ever inform a strategy around protecting a company. But looking for trends in those, looking for what a lot of people are talking about and realizing that that might be a priority to ask the security team, to ask your own business. That does make a lot of sense. So if you see a common thread amongst five or six different forums and you don’t know what you should be doing to react to it, that’s absolutely a valid strategy. The other thing that can be really valuable is the vendors themselves. And so, you know, of course, we’re all being inundated by sales, marketing emails on a constant basis. But allowing a few of those in the door and asking them directly, how are you going to help secure my company? You can learn some things by how those vendors and technologies were used in your peer groups and other companies. they are helping people do this. Otherwise, they wouldn’t be in business. And so sitting down and listening to those. But again, not listening to one. No one vendor is going to come in and solve all the world’s problems. And so being able to look for the commonality of threads, look for the understanding of what they’re saying, piece together your strategy for your company, that’s really the right way to think about it.

Speaker 0 | 15:19.734

So you have another partner who I won’t name his name because he’s kind of behind the iron curtain. And no matter how hard I tried to ask to have him on the show, Also, you guys would not allow him. So I don’t know if it’s because he’s hiding behind the iron curtain. We’ll just assume that. But I wanted to have both you guys on. But I listened to a talk he gave about a couple weeks ago, and he just mentioned some of the busy times of year, particular holidays where you guys get hit with massive attacks or you have to bring in special forces last minute to help a company that’s maybe under, you know, getting completely shut down by a massive DDoS attack or whatever it is. Maybe just, you got any good stories there around that? What is CenturyLink doing last minute to save a company that’s under attack? Maybe the best question there is, what do you do when it’s too late and you’ve just been completely destroyed by something?

Speaker 1 | 16:17.808

Yeah, so as a company, we’ll take the DDoS example you gave. From a capacity perspective, we’re not… particularly concerned with the attack against our customers that are buying that service right we have plenty of capacity to ingest the attack volume to mitigate it and and make sure the customer stays up but you’re right there are attacks that hit companies that don’t believe they’ll be targets of attack there are companies that maybe haven’t provisioned the entirety of their infrastructure behind a service like ours and so you’re right we do have customers come to us and say I’m down And that’s not an uncommon thing that we get from our customer base. And so there’s a few things that we do. The first is we do use that threat intelligence data that I’ve spoken about. We go back to that and we say, do we see the attack? We actually stay connected to these botnets and log what they are doing and who they are attacking. We go in and we try to understand, do we see it? And if we do, that actually helps because now we can go and describe who. might be behind the attack. What other things do they attack? That can help us understand the motivation for that customer and help them understand, do they need to all hands on deck move to procuring a DDoS service today? Or is this a one-off attack that, you know, maybe they have a few weeks to think about it and rationalize how they should protect their business. But in the instances where they are down hard and they cannot get back online, absolutely, we will essentially move their internet connection behind that. big ingestion infrastructure I described and mitigate the attack and work with them on it to get them back online. Some of the context of the question you asked is, are there times of year that this happens? Are there times where we have to pull in extra people or be hyper attentive? And absolutely, right? So let’s look back to the motivations of the people behind these attacks. And they sort of fit into two categories. One is fame and recognition. So at the lowest end of the sophistication, there’s some folks that want to show off to their gaming friends. They want to show how powerful they are in whatever forum or whatever Discord chat room they’re in. They want to be the powerful one and be respected. So they’re looking for fame. Well, there’s times of year where knocking down a website gets you a little more attention. And obviously between Thanksgiving and Christmas here in the U.S. and a lot of the world is a high time for making money on a website. And if one of these actors could take down a website, they’re going to get extra attention during that time of year. So that is the time of year where we make sure we’re, like I said, hyper-attentive. We’re paying attention to what’s going on, looking for trends in the attacks and trying to stay ahead of both the attacks, as well as if we see any botnet grow a little more aggressively than others, we will be more aggressive about taking it down and impacting its ability to operate. The other time of year… is, and this is sort of funny, is when people aren’t in school, right? So the college students, the high school students, they tend to be the ones that personify that fame motivation. And when they’re not in school, they have more idle time. So during the summer, we see a lot of attacks from those actors as well. The other motivation is criminals. And so how do you make the most money? And so from that bucket, they, of course, ransomware, particularly popular during the time of year where people would be making more revenue and more income in their business. If you can hold a company ransom, they’re more likely to pay in order to get back into operation during that time period. And so, you know, different times of year, different events. And then the other thing that drives that is new exploits. And so staying on top of understanding when new bugs are being introduced, when vendors are filling out new patches to protect against new known vulnerabilities, actors will step up and take advantage of that because they have a very finite time window where we’ll call it the Western economy will pass. Very few businesses will pack inside of two hours. And so the faster they can create an exploit to deliver their malicious payload, the more infrastructure on the internet they’re going to be able to gain control of and then build their botnets and carry out their financial goals.

Speaker 0 | 20:43.991

What are some of the things that CenturyLink specifically is doing? So I guess ultimately, what are you guys providing that helps with various different issues? How does… How does someone engage you? How does someone engage your team? Maybe just talk a little bit about some of the, you know, how we go through qualifying various different issues and problems that a company would have and how CenturyLink can help.

Speaker 1 | 21:08.686

Yeah, great question. So CenturyLink offers a variety of security services to help our customers. And it really doesn’t matter where they are in their evolution. We’re ready to have that engaging conversation with them. And so from the least sophisticated. security program or at least mature security program, we have consultants that can come in and help build the program from scratch. And for a lot of companies, that’s where they are. They need that help. And so there’s nothing wrong with reaching out and asking for help. And we’re able to engage. And that could be as simple as policy reviews. Are the right risk levels being addressed inside the company through what is that control structure and how do you actually protect the company? How do you monitor it? Things like that. Which then leads us into sort of the repeatable product sets that we offer to customers. And so we offer what we call our adaptive network security product that is a collection of network-based controls. So firewall, IPS, DLP, sandboxing, remote VPN access, just about everything that you can think of being provided to protect an enterprise and the network is included in that product set. And we see that one being very popular because a lot of companies, they end up buying six different devices at their premises in order to provide that for themselves. And that gets on earth to run. It’s costly from a maintenance perspective. And so that product set helps with that area. The watching side that I described earlier, you recall, you know, inventory controls and then monitoring. We have a security log monitoring platform. that does the event correlations, does the alarming, includes some easy to use mobile apps, things like that in order to help companies what’s going on inside their environment and help stay abreast of where they should be reacting, what are the threats specific to them. And then DDoS mitigation, of course, we’ve been speaking about quite a bit, where we help protect the infrastructure online. And then the threat intelligence product set, where we provide analytics and data about where we see maliciousness interacting with their infrastructure? What’s it doing? How is it operating? More information on how they could clean up and mitigate or stop what’s going on in terms of that communication.

Speaker 0 | 23:33.678

Now, with all the flashy lights and bulbs, as Chris Roberts likes to say on LinkedIn a lot, he says there’s a lot of flashy lights out there in the security world. How does one even begin to… you know, differentiate between the thousands of security products? Is there anything that you can, is there anything that you can provide there of that’s, you know, would make it easier for someone to a trust you guys, uh, put their security in your hands, maybe, maybe a great, uh, you know, give me some of your success stories. Maybe.

Speaker 1 | 24:05.641

Yeah. The number one recommendation I have on, on how to get past the blinky lights problem is a company needs to decide, are they going to be in charge of their controls? or are they going to hold someone else accountable to being in charge of their control? And so if a company decides that they are going to maintain all of that infrastructure and all the overhead and the monitoring that comes with those blinky lights, then they need to spend a lot of time hiring staff, maintaining staff, which we all know is very difficult in the security market, and being able to not miss things. And so where we come in is we say, great. ask us how we’re going to secure your infrastructure. How are we going to operate your controls? How are we going to make sure we don’t miss things? And so that’s the challenge that, you know, customers could be giving to us. So that they don’t have to worry about which blinky light’s on or off, or which next box they have to go plug into their data center. That’s our problem, and they just need to keep asking us, how are you going to protect us? How are you going to mitigate those problems? And so one of the things that comes with buying a service like that is the security operations center we staff that sits behind it. And so we’ve got a global security operations staff sitting on three continents, and they are watching our customer base. And they get benefit not only of course scale and the amount of customers they’re able to watch, but when one customer is attacked by something, they’re able to use that knowledge for the rest of the customer base. And so what could look like an innocuous alarm in the event correlation system might actually be contextually more interesting based on what’s going on on the internet. And so that’s what we challenge our customers with is ask us how we’re going to secure it for you. How are we going to operate those controls? How are we going to make it so you don’t have to worry about the next box you’re plugging in? And if the customer is in a position where they understand their inventory, they can have a really mature conversation with us.

Speaker 0 | 26:05.353

That’s awesome. Yeah, where someone’s, any company might be a big company, but you’re a very big company in the fact that you handle multiple, multiple companies. So the fact that you handle, you know, issues at scale, whereas one company can only handle the issues that hit them. You’re handling issues at scale so that you get the benefit of the family, let’s say. Mike, it’s been great talking to you, man. I really appreciate it. The conversation’s been, I could probably ask you a thousand more questions before we end. If you had one final message you had to deliver to anyone listening out there, what would that be?

Speaker 1 | 26:46.038

Yeah, the one message I like to leave the security world with is don’t look at one aspect. protecting things. Look at all of it and figure out how to maturely create programs, cycles, iterative processes out of it. So I mentioned watch it, inventory it, block it. That’s an example of that. And people are quick to ignore aspects of the security industry that maybe don’t fit into their current program, their current capabilities, and that could be to their disadvantage. But one of the things that I like to consider for folks is to look at threat intelligence. So a lot of folks are struggling to understand how threat intelligence helps protect their company, how it actually helps them block things. And so back to the monitoring goal, you know, how are you going to watch your company? Well, one of the things you’re going to watch is how your controls are alarming, reporting, what events are happening. But it would be great if you could get ahead of some of that by knowing where the malicious infrastructure was. elevating those along. And so, you know, that’s an example of an area that’s been gaining a lot of maturity in the security industry, but people are really struggling how to get value out of it. And I think figuring out how to get the appropriate value in terms of knowing where to look, knowing how to tie that into their SIM or their hosted SIM solution that they have, I think it’s an important aspect of protecting a company. And it can’t be in the place of, of course, controls, blocking things. understanding where your stuff is. And then all of the blocking and tackling that I hope folks know, which is make sure you pack, make sure you scan for known vulnerabilities, make sure you validate your configs, things like that. So if there was anything I would leave the listeners with is make sure you look at all aspects of the security industry. They exist for a reason, and understanding how to use them effectively is an important part of maturing.

Speaker 0 | 28:44.410

Awesome. Thank you so much. CNSG obviously has had a great relationship with… CenturyLink for a long time. So if there’s anyone out there listening that would like to set up a, you know, an initial meeting to go over your security landscape at your company, certainly reach out to us on LinkedIn. Have a great day, man. Thanks a lot.

Speaker 1 | 29:01.810

All right. Thank you. Have a good rest of your day.

15. DDoS Attacks, Botnets, & Security Overwhelm.

Speaker 0 | 00:00.544

Welcome everyone to Telecom Radio 1 and today we have a very, very complicated subject. I don’t even know where to begin with this. It’s called security. We’re not talking about people in hoodies hiding behind computer screens or anything. We’re actually going to delve in with someone that is very knowledgeable on this subject. Today we have Mike Benjamin on from CenturyLink. Mike, welcome to the show. Thank you very much.

Speaker 1 | 00:25.824

Yeah, thanks for having me, Phil. Looking forward to talking about security. Yeah.

Speaker 0 | 00:29.427

Now- You have been described, you know, I have my own description, but you’ve been described as the man that has been hiding behind the iron curtain for years. I like to describe you as the most popular unknown man in security. And that’s for a very good reason, because we don’t really want the very knowledgeable people to be known and, you know, where they are and what they’re doing, because we don’t want them to come after you. Actually, I don’t really know what that is, because… security is not supposed to be all fear-mongering. We’re actually supposed to have people that know what they’re doing. But give me a little bit of background on yourself, maybe a great story. Just tell me a little bit about what you’ve been doing for the last 10 years.

Speaker 1 | 01:08.780

Yeah, absolutely. So I lead a team here at CenturyLink called the Threat Research Labs Group. And you’re right, as we were building this team, we made a real focused decision to not name individuals from the group. We did that for a variety of reasons. One, as any group knows, You want to aim for group success, not individual success. So that was an easy one just for morale of our team. That’s the sort of less interesting side as opposed to the audience. But realistically, when we were out there, we knew that one of the goals for our group was to break criminal infrastructure, to take down botnets, to impact their ability to actually operate. And we knew just inevitably we’d anger some people. So why publicize the individuals from the group? You know, that didn’t make a lot of sense. to us. And so as we’ve progressed, we’ve done a lot of work with other security companies. We’ve done a lot of work with the industry and the infrastructure providers of the internet to literally clean up what’s going on that’s bad on the internet. And we believe we’ve made a material impact. And so, you know, you may have mentioned to me, you know, hiding, so to speak. You know, realistically, we had a reason to now talk more publicly about it. And it’s, we want to get more people involved. We want to get more of the internet, more of the… operators of that infrastructure to help us impact the infrastructure, clean up the internet, and collaborate on what, like you described, is a massive problem that a lot of people don’t know where to even start with. And so we’ve been working for, we’ll call it the last three years, really specifically on one particular problem I’ll describe to you, which is the DDoS botnet space. And so there was a lot of press a couple years ago with the Mirai malware family causing attacks that were… impacting some pretty big infrastructure and big websites. And we were fortunate enough to be in a position to have a lot of visibility of what was going on and have had a lot of impact on minimizing it since that time period. And so we were able to pull together some working groups. We were able to join some other working groups of people that had a like-minded ability to clean up the Internet, had time in their day. And in a lot of cases, I’ll tell you, these are people who do security work during the day, help their company operate. and then they go home and they have a real interest in doing this with us off hours. So some of them have brought that back to work. Some of them just continue to do it off hours. And so over the time period that we’ve been doing it, we’ve watched hundreds and hundreds of instances of these botnets stand up, and we’ve worked to take them down. We actually had a funny story where one of them actually registered a domain, one of the operators that contained the F word and then our company name. as the domain they put their command and control, their botnet on. So we’d made them so mad that they decided to laugh down at us a little and, you know, essentially tell us to step off in regards to that. So it was pretty satisfying for the group to see, you know, we knew we were making an impact on their day as we saw that register.

Speaker 0 | 04:12.036

Nice. Were you able to take down that botnet?

Speaker 1 | 04:15.078

We were. We were. I actually tried to come up, you know, I’m not a lawyer, but I tried to come up with… some legal reason why we can actually take ownership of the domain and redirect it to our security website. I couldn’t convince anybody of that. But yeah, we were able to take it down just like we do the others.

Speaker 0 | 04:30.931

Nice, nice. So maybe just for the layman listening and like myself, and just for other people that might be listening that don’t know exactly what a DDoS attack is or what a botnet is, maybe just give us just the general, you know, give us the general overview for anyone that might be listening that has no clue what that is.

Speaker 1 | 04:49.992

Yeah, absolutely. So we can imagine criminals want to do bad stuff to people, but they don’t want to use their own home computer. That’s a little too easy to find. It’s a little too easy to track, a little too easy to understand. And so they want to obtain other infrastructure to do it from. And so they use a lot of different ways. In some cases, they just even go out and buy virtual machines. They may even pay money. That money may not be so legally obtained in the first place, but in other cases, they break into computers. They also try to infect people with malware where they can control the computer. And depending on their scope and goals, They want as much infrastructure as possible in order to launch these attacks. And in the case of botnets, what we see is what they’ve done is they’ve said, you know, whatever they’re trying to carry out, we see a botnet deployed for spam. We see it for denial of service attacks. We see it for data exfiltration. They want a lot of computers in order to make the maximum impact on what they’re trying to accomplish. So DDoS specifically to your question. is at its core, it’s a denial of service. And you think what those words mean is it overwhelms something, so it can’t do its core purpose. So in the most simple terms, if somebody has a gigabit Ethernet loop to their office, if the bad guys can send over a gigabit of traffic to that connection, obviously the connection can’t do anything real with that capacity. In other cases, it could be a web server. A web server might be configured, you know, if somebody stands up… taxi on their Linux machine and says, hey, I want to serve up to a thousand connections a second. Well, in that case, bad guys might just send a thousand and one connections. There you go. The web server can’t do what it’s supposed to do at its core. And so that’s what the denial of service aims to do. However, as internet infrastructure has improved, the authentication services have matured, it’s become harder and harder to actually carry out what people aim to do on the criminal side. And so they started to distribute the attack. So rather than buying one VM and launching an attack or breaking into two or three servers and launching it, in some cases they want thousands, tens of thousands, hundreds of thousands. We’ve seen over a million in some instances computers on the Internet in which to launch these attacks. And if you can imagine what a million cable modems could generate on traffic, it’s pretty substantive. And so people’s homes even end up being participants in this, as we see in the IoT DDoS botnet space.

Speaker 0 | 07:22.911

Especially if they’ve got a 400 meg connection that they don’t need at their house for two people. But it’s interesting. I was reading your security report. And you guys had, from 2017, you said that there was 1.8 million. You had the top five bot hosting countries. And you had 1.8 million as a number next to the United States. And then you had China at 454,000. Now, that was a daily average. So are you saying that 1.8 million devices in the United States are being taken over and or used actively on a daily basis?

Speaker 1 | 07:59.532

Yeah, that’s what those numbers represent. And so what we’re utilizing is the intelligence data we’ve been able to collect from a variety of sources, from watching the botnets operate, tearing apart their malware, allowing them to attack us inside of Honeypots and other things that are intentionally built to be attacked. And we put together a reputation system with an understanding of where threats are in the Internet. And so the understanding of a malware family and what the main control infrastructure of a malware family is, is known to a lot of threat intelligence companies. Where we’ve taken that data a step further, and that’s where those numbers come from that you’re quoting, is we can, from a network perspective, actually look and see what is being controlled by those hosts. So if somebody installs a piece of malware, If you were to just look at the network connectivity, you could say, hey, it talks to that server. That server must be bad. But you don’t really have an understanding of is that server controlling 10 computers, 100,000 computers? What are they doing? How often are they hacking? And that’s the kind of information we aim to glean from that larger data set.

Speaker 0 | 09:07.216

Awesome. How much knowledge would you say the typical IT director, maybe CTO or CIO, how much? How much typical security knowledge does an IT director have, would you say, that you guys see just in general, and how much help do they need in the security realm?

Speaker 1 | 09:29.549

Well, the easy answer is it varies pretty widely, but I’ll try and quantify that a little bit more. So at the largest of enterprises…

Speaker 0 | 09:39.074

It’s very vague. I get that it’s a very vague question. Maybe I should be more specific around… around the botnet DDoS attacks. Obviously, typically, your IT director is like, hey, I can manage my firewall. I manage my network. They’ve got their… their general strengths let’s just put it this way what would you say is a let me ask it this way what is a typical weakness or what is a a typical uh area of of knowledge or expertise where people are lacking so that’s that’s a really good question the

Speaker 1 | 10:15.617

area that i see uh your you know sort of average business having the most common weakness across them is knowledge of their own environment so if you look at the security industry there’s a lot of folks that are from a consumer of that technology, right? They’re out and they’re trying to buy a better appliance. They’re trying to buy a better piece of software. And what they’re trying to augment is really their control. So your controls in the security world are the things that block stuff or minimize the ability to do stuff. And so antivirus is a control. A firewall is a control. Everything that sits in that sort of technology vertical largely is a control. So we see a lot of companies out there investing a lot of money in buying controls, and that’s good, right? That means they can block more stuff. That means they have more ability and more places to inspect and look for the good versus the bad. The problem we see, however, is they don’t know what they’re protecting. So if you take a step back and you think about your average company, how good is their actual inventory? Do they actually know where all the servers are, all the applications? And as we’ve seen recently, as we’ve seen libraries, become exploited? Do they understand what they’re writing software against? What are they linking into their applications that they are developing, whether it be a public or internal? They don’t know what they have. And so you can spend all the money in the world and buy every control on earth. If you don’t put it in the right places and turn on the right mitigations and understand your inventory adequately, they’re not going to do what they need to do. So that’s the number one thing we see. The second, though, is security. And this is a… a sad thing to say as an industry, but ultimately you can’t stop everything. No security solution on earth is a hundred percent effective. And so with that knowledge in mind and with that realization, people need to also be looking at how do they log what’s going on and how do they watch and then how do they react? And so that’s where that intelligence part feeds in is if people have an understanding of where their stuff is, they’ve put the right controls in. and then they watch what’s happening to it. They can then react and minimize when that inevitable break-in does happen because they can isolate to one server, keep it to five minutes. Therefore, the actor’s not been able to carry out on whatever goals they have. And so recommendations to a business looking at security is if they’re not asking all three of those questions, then they are not securing their company adequately. So where’s their stuff? How are they deploying controls to stop it? And then how are they monitoring at the end of the day?

Speaker 0 | 12:54.924

I had a follow-up question here. it’s kind of really off in left field, but I find that a lot of people, when they don’t have knowledge on a subject, they typically go to the internet and start searching for it. So as a complete side note, let’s just say you’re in charge of your entire company, your IT director, and you don’t have an answer to something. I find a lot of people going to various IT forums, okay? I just want, I just… I just want to ask you, what do you think of Reddit, maybe Spiceworks, some other IT forums? Should people trust these groups all the time? Should they be going and getting their information there? Just curious, complete side note, side question.

Speaker 1 | 13:40.140

It is a great question. Interestingly enough, people actually ask me, how do I keep abreast of the security market, the news, the things going on? And I tell them it’s a combination of this. massive rats nest of RSS feeds I’ve compiled over the years and Twitter. And so I do also subscribe to the NetSec subreddit and I do read that. But realistically, no one news source could ever inform a strategy around protecting a company. But looking for trends in those, looking for what a lot of people are talking about and realizing that that might be a priority to ask the security team, to ask your own business. That does make a lot of sense. So if you see a common thread amongst five or six different forums and you don’t know what you should be doing to react to it, that’s absolutely a valid strategy. The other thing that can be really valuable is the vendors themselves. And so, you know, of course, we’re all being inundated by sales, marketing emails on a constant basis. But allowing a few of those in the door and asking them directly, how are you going to help secure my company? You can learn some things by how those vendors and technologies were used in your peer groups and other companies. they are helping people do this. Otherwise, they wouldn’t be in business. And so sitting down and listening to those. But again, not listening to one. No one vendor is going to come in and solve all the world’s problems. And so being able to look for the commonality of threads, look for the understanding of what they’re saying, piece together your strategy for your company, that’s really the right way to think about it.

Speaker 0 | 15:19.734

So you have another partner who I won’t name his name because he’s kind of behind the iron curtain. And no matter how hard I tried to ask to have him on the show, Also, you guys would not allow him. So I don’t know if it’s because he’s hiding behind the iron curtain. We’ll just assume that. But I wanted to have both you guys on. But I listened to a talk he gave about a couple weeks ago, and he just mentioned some of the busy times of year, particular holidays where you guys get hit with massive attacks or you have to bring in special forces last minute to help a company that’s maybe under, you know, getting completely shut down by a massive DDoS attack or whatever it is. Maybe just, you got any good stories there around that? What is CenturyLink doing last minute to save a company that’s under attack? Maybe the best question there is, what do you do when it’s too late and you’ve just been completely destroyed by something?

Speaker 1 | 16:17.808

Yeah, so as a company, we’ll take the DDoS example you gave. From a capacity perspective, we’re not… particularly concerned with the attack against our customers that are buying that service right we have plenty of capacity to ingest the attack volume to mitigate it and and make sure the customer stays up but you’re right there are attacks that hit companies that don’t believe they’ll be targets of attack there are companies that maybe haven’t provisioned the entirety of their infrastructure behind a service like ours and so you’re right we do have customers come to us and say I’m down And that’s not an uncommon thing that we get from our customer base. And so there’s a few things that we do. The first is we do use that threat intelligence data that I’ve spoken about. We go back to that and we say, do we see the attack? We actually stay connected to these botnets and log what they are doing and who they are attacking. We go in and we try to understand, do we see it? And if we do, that actually helps because now we can go and describe who. might be behind the attack. What other things do they attack? That can help us understand the motivation for that customer and help them understand, do they need to all hands on deck move to procuring a DDoS service today? Or is this a one-off attack that, you know, maybe they have a few weeks to think about it and rationalize how they should protect their business. But in the instances where they are down hard and they cannot get back online, absolutely, we will essentially move their internet connection behind that. big ingestion infrastructure I described and mitigate the attack and work with them on it to get them back online. Some of the context of the question you asked is, are there times of year that this happens? Are there times where we have to pull in extra people or be hyper attentive? And absolutely, right? So let’s look back to the motivations of the people behind these attacks. And they sort of fit into two categories. One is fame and recognition. So at the lowest end of the sophistication, there’s some folks that want to show off to their gaming friends. They want to show how powerful they are in whatever forum or whatever Discord chat room they’re in. They want to be the powerful one and be respected. So they’re looking for fame. Well, there’s times of year where knocking down a website gets you a little more attention. And obviously between Thanksgiving and Christmas here in the U.S. and a lot of the world is a high time for making money on a website. And if one of these actors could take down a website, they’re going to get extra attention during that time of year. So that is the time of year where we make sure we’re, like I said, hyper-attentive. We’re paying attention to what’s going on, looking for trends in the attacks and trying to stay ahead of both the attacks, as well as if we see any botnet grow a little more aggressively than others, we will be more aggressive about taking it down and impacting its ability to operate. The other time of year… is, and this is sort of funny, is when people aren’t in school, right? So the college students, the high school students, they tend to be the ones that personify that fame motivation. And when they’re not in school, they have more idle time. So during the summer, we see a lot of attacks from those actors as well. The other motivation is criminals. And so how do you make the most money? And so from that bucket, they, of course, ransomware, particularly popular during the time of year where people would be making more revenue and more income in their business. If you can hold a company ransom, they’re more likely to pay in order to get back into operation during that time period. And so, you know, different times of year, different events. And then the other thing that drives that is new exploits. And so staying on top of understanding when new bugs are being introduced, when vendors are filling out new patches to protect against new known vulnerabilities, actors will step up and take advantage of that because they have a very finite time window where we’ll call it the Western economy will pass. Very few businesses will pack inside of two hours. And so the faster they can create an exploit to deliver their malicious payload, the more infrastructure on the internet they’re going to be able to gain control of and then build their botnets and carry out their financial goals.

Speaker 0 | 20:43.991

What are some of the things that CenturyLink specifically is doing? So I guess ultimately, what are you guys providing that helps with various different issues? How does… How does someone engage you? How does someone engage your team? Maybe just talk a little bit about some of the, you know, how we go through qualifying various different issues and problems that a company would have and how CenturyLink can help.

Speaker 1 | 21:08.686

Yeah, great question. So CenturyLink offers a variety of security services to help our customers. And it really doesn’t matter where they are in their evolution. We’re ready to have that engaging conversation with them. And so from the least sophisticated. security program or at least mature security program, we have consultants that can come in and help build the program from scratch. And for a lot of companies, that’s where they are. They need that help. And so there’s nothing wrong with reaching out and asking for help. And we’re able to engage. And that could be as simple as policy reviews. Are the right risk levels being addressed inside the company through what is that control structure and how do you actually protect the company? How do you monitor it? Things like that. Which then leads us into sort of the repeatable product sets that we offer to customers. And so we offer what we call our adaptive network security product that is a collection of network-based controls. So firewall, IPS, DLP, sandboxing, remote VPN access, just about everything that you can think of being provided to protect an enterprise and the network is included in that product set. And we see that one being very popular because a lot of companies, they end up buying six different devices at their premises in order to provide that for themselves. And that gets on earth to run. It’s costly from a maintenance perspective. And so that product set helps with that area. The watching side that I described earlier, you recall, you know, inventory controls and then monitoring. We have a security log monitoring platform. that does the event correlations, does the alarming, includes some easy to use mobile apps, things like that in order to help companies what’s going on inside their environment and help stay abreast of where they should be reacting, what are the threats specific to them. And then DDoS mitigation, of course, we’ve been speaking about quite a bit, where we help protect the infrastructure online. And then the threat intelligence product set, where we provide analytics and data about where we see maliciousness interacting with their infrastructure? What’s it doing? How is it operating? More information on how they could clean up and mitigate or stop what’s going on in terms of that communication.

Speaker 0 | 23:33.678

Now, with all the flashy lights and bulbs, as Chris Roberts likes to say on LinkedIn a lot, he says there’s a lot of flashy lights out there in the security world. How does one even begin to… you know, differentiate between the thousands of security products? Is there anything that you can, is there anything that you can provide there of that’s, you know, would make it easier for someone to a trust you guys, uh, put their security in your hands, maybe, maybe a great, uh, you know, give me some of your success stories. Maybe.

Speaker 1 | 24:05.641

Yeah. The number one recommendation I have on, on how to get past the blinky lights problem is a company needs to decide, are they going to be in charge of their controls? or are they going to hold someone else accountable to being in charge of their control? And so if a company decides that they are going to maintain all of that infrastructure and all the overhead and the monitoring that comes with those blinky lights, then they need to spend a lot of time hiring staff, maintaining staff, which we all know is very difficult in the security market, and being able to not miss things. And so where we come in is we say, great. ask us how we’re going to secure your infrastructure. How are we going to operate your controls? How are we going to make sure we don’t miss things? And so that’s the challenge that, you know, customers could be giving to us. So that they don’t have to worry about which blinky light’s on or off, or which next box they have to go plug into their data center. That’s our problem, and they just need to keep asking us, how are you going to protect us? How are you going to mitigate those problems? And so one of the things that comes with buying a service like that is the security operations center we staff that sits behind it. And so we’ve got a global security operations staff sitting on three continents, and they are watching our customer base. And they get benefit not only of course scale and the amount of customers they’re able to watch, but when one customer is attacked by something, they’re able to use that knowledge for the rest of the customer base. And so what could look like an innocuous alarm in the event correlation system might actually be contextually more interesting based on what’s going on on the internet. And so that’s what we challenge our customers with is ask us how we’re going to secure it for you. How are we going to operate those controls? How are we going to make it so you don’t have to worry about the next box you’re plugging in? And if the customer is in a position where they understand their inventory, they can have a really mature conversation with us.

Speaker 0 | 26:05.353

That’s awesome. Yeah, where someone’s, any company might be a big company, but you’re a very big company in the fact that you handle multiple, multiple companies. So the fact that you handle, you know, issues at scale, whereas one company can only handle the issues that hit them. You’re handling issues at scale so that you get the benefit of the family, let’s say. Mike, it’s been great talking to you, man. I really appreciate it. The conversation’s been, I could probably ask you a thousand more questions before we end. If you had one final message you had to deliver to anyone listening out there, what would that be?

Speaker 1 | 26:46.038

Yeah, the one message I like to leave the security world with is don’t look at one aspect. protecting things. Look at all of it and figure out how to maturely create programs, cycles, iterative processes out of it. So I mentioned watch it, inventory it, block it. That’s an example of that. And people are quick to ignore aspects of the security industry that maybe don’t fit into their current program, their current capabilities, and that could be to their disadvantage. But one of the things that I like to consider for folks is to look at threat intelligence. So a lot of folks are struggling to understand how threat intelligence helps protect their company, how it actually helps them block things. And so back to the monitoring goal, you know, how are you going to watch your company? Well, one of the things you’re going to watch is how your controls are alarming, reporting, what events are happening. But it would be great if you could get ahead of some of that by knowing where the malicious infrastructure was. elevating those along. And so, you know, that’s an example of an area that’s been gaining a lot of maturity in the security industry, but people are really struggling how to get value out of it. And I think figuring out how to get the appropriate value in terms of knowing where to look, knowing how to tie that into their SIM or their hosted SIM solution that they have, I think it’s an important aspect of protecting a company. And it can’t be in the place of, of course, controls, blocking things. understanding where your stuff is. And then all of the blocking and tackling that I hope folks know, which is make sure you pack, make sure you scan for known vulnerabilities, make sure you validate your configs, things like that. So if there was anything I would leave the listeners with is make sure you look at all aspects of the security industry. They exist for a reason, and understanding how to use them effectively is an important part of maturing.

Speaker 0 | 28:44.410

Awesome. Thank you so much. CNSG obviously has had a great relationship with… CenturyLink for a long time. So if there’s anyone out there listening that would like to set up a, you know, an initial meeting to go over your security landscape at your company, certainly reach out to us on LinkedIn. Have a great day, man. Thanks a lot.

Speaker 1 | 29:01.810

All right. Thank you. Have a good rest of your day.

Share This Episode On:

HOSTED BY PHIL HOWARD

Dissecting Popular IT Nerds Podcast

Weekly strategic insights from technology executives who understand your challenges

Are You The Nerd We're Looking For?

ATTENTION IT EXECUTIVES: Your advice and unique stories are invaluable to us. Help us by taking this quiz. You’ll gain recognition good for your career and you’ll contribute value to your fellow IT peers.

QR Code