My friend, Dave, is a New York State police officer, and when I go to his house for barbecues, I enjoy listening to his stories while we sit around the campfire.
At one particular cookout this past summer, he recounted some recent excitement.
“We raided that drug lab a few times before, so we knew what we were getting into. Tommy busted in the door and we went in with local police providing cover.”
Wow. I think I heard about that one on the news.
When he finished, it felt like it was my turn to say something. I started by making a joke about how I just mess around with computers all day. He looked at me with a solemn face and explained, “You don’t want to go raiding drug labs, Phil. Be grateful for what you have.”
Kind words, but I still felt like Gomer Pyle talking to Rambo. Nevertheless, he insisted that the majority of his time is spent at a desk, in a cubicle, typing reports.
“Phil,” he started, “ninety-nine percent of the time I’m sitting around doing boring, tedious work, waiting for something exciting to happen. And when it finally does, it’s over in a flash.”
This resonated with me. You see, I really loved being a traditional network engineer. Maybe it was because I worked for value-added resellers and got to put my hands on the latest and coolest new gear, but I think it was more because I had the opportunity to fly from project to project as the network hero. Every day felt like excitement, organized chaos, and adventure.
Network security is a very different ballgame, though. Today I work for an internal IT department, not a consulting firm, and instead of going from project to project as the network hero, I spend my days in the tedium and monotony of security tasks. Gathering and poring over huge amounts of information from network scans and audits isn’t exciting at all, but this tedium is the heart of information security.
Dave explained it to me. In law enforcement, collecting and categorizing information that might lead to an arrest and ultimately become evidence in court is of paramount importance. Nothing can be missed, and everything must be done perfectly. The exciting part might be only a few minutes, but the tedious work leading up to and as a result of it is just as important.
Information security has many parallels. One of the reasons the work I do is so dull is that, well, it’s just so dull. Managing scripts to gather data, overseeing a handful of scanning platforms that generate a ton of noise, and looking at the audit summary emails that inundate my mailbox every morning doesn’t motivate me to get out of bed each day. Security fatigue is a real thing.
But then it happened. As our conversation shifted to talking about our kids, my phone began buzzing frantically. I fumbled with the unlock button in the dark only to see that it was my boss texting me that file servers were locked up. It looked like a ransomware attack, and I knew why he was so worried. These servers had a ton of valuable intellectual property on them.
Immediately I got to my feet and apologized for having to leave in a hurry. Normally I’d just VPN into the corporate network, but I didn’t have my laptop and Dave’s satellite Internet connection was abysmal. Besides, in a situation like this I wanted as many monitors in front of me as possible as well as the big screens on the wall displaying security monitoring dashboards in real time.
After getting to the office, segregating the site from the rest of the global network was the first step. The high-level plan was simple: mitigate the risk to the rest of the network and stop the bleeding
After a small team assembled and checked for ransomware on the rest of the network, things settled down. The CIO called looking for an update. Did we lose any data? How did this ransomware get in? How did it spread? Did we have good backups?
This is where the rubber meets the road in information security. When everything is locked down and the bleeding has been stopped, the questions begin. Just like that quick burst of intense action followed by a return to the mundane that my friend described around the campfire, this was the time for me to settle in and hunt down clues in the myriad of scan data.
But keep in mind that this is no trivial task. No automated scripts kicked off when anomalous activity was detected, and I had multiple tools to look at in order to get a complete picture. I had to look at logs. I had to scroll through dozens of graphs. I had to wake people up late at night. I had to log into dozens of devices. If only I had at my fingertips the information of what went on right at the time of the attack.
This is tedium, this is inefficiency, this is error-prone, and this is certainly not the way to manage a cyberattack in the 21st century. But it’s what I had, and I needed to trace how this malware got in and made its way to the file servers.
Conversely, NetBrain automates this entire process. In my particular case, Executable Runbooks could have been kicked off automatically to run scripts gathering information in real time. That would have put specific and relevant information right at our team’s fingertips when we needed it.
This isn’t a minor advantage in a security post-mortem. I had to go it alone, searching through logs generated from devices managed by multiple teams. Automated Runbook outputs would have made available to all our teams the same relevant information at the same time.
And just as important as finding out how and when the bad guys got in is how the malware spread through our network. NetBrain Dynamic Mapping provides the visual troubleshooting that I had to manually figure out. Instead of crawling devices one by one, which is both tedious and error-prone, NetBrain provides an API-triggered method for dynamically creating a network map of an attack path in real time.
We weren’t thoroughly prepared for an attack mostly because of the time and energy it takes to be prepared and respond appropriately. Robust software that automates the entire process would have empowered me to dramatically improve my company’s security posture without having to hire teams of interns to read log files.
Unfortunately, though, I was stuck in the Stone Age. After calling my wife to let her know I’d be a while, I poured myself fresh cup of coffee, took a deep breath, and started digging through logs manually . . . one line at a time.