I was recording Defense in Depth with David Spark, with special guest Yoav Alon, and David made a comment about why people hide mistakes, along the lines of “well, a cloud oops can get really big and nasty, and who wants to be blamed for that?”
The only thing worse than an admitted mistake that burns your business is an unadmitted mistake that burns your business. If there is a bad outcome, of course there will be a post-mortem, and of course the company will find out you triggered it. Whatever bad outcome you were afraid of? You’ll get that, and more. And by not pointing at it as soon as you knew it was a problem, you made triage and recovery harder for your company. In this blog post, I want to unpack some key ideas around making mistakes as they relate to provisioning cloud resources and ensuring they’re secure, in order to spark conversation around the human element and culture around cybersecurity issues.
Psychology and People: Key in Any Business Setting
I know there exists a school of thought that believes that “humans are the weakest link,” and that if only those pesky humans weren’t causing problems, security would be simple. I’m a fan of Nancy Leveson, one of the great minds in complex systems safety, who says, “Human error is a symptom of a system in need of redesign.” We should be celebrating humans who inadvertently trigger serious design issues, so that we can fix them. If you’re the human who triggered it, I hope you’re in an environment that won’t punish you. But even if you are, you should still volunteer the fact that you’re the one that made the mistake, so your org can clean up the mess as quickly as possible.
When we apply this concept to cloud security, something that often comes to mind but is not talked about from this perspective is, you guessed it: DevSecOps. Yet another term thrown around the cybersecurity jargon arena as it may be, DevSecOps is fundamental when it comes to recognizing that errors can and do happen—and often. By encouraging a culture of DevSecOps, we bring forth the idea that we can catch security errors (read: mistakes) early before they become out of control. By definition, DevSecOps recognizes that there not only can be security issues, but that there will be security issues, and it’s everyone’s job to ensure security is woven in at every stage of the software development lifecycle (SDLC) to keep glaring, code-red security issues to a minimum.
Where Does the Shared Responsibility Model Come into Play?
Let’s take a short but necessary detour and shine light on the fact that again, cybersecurity mistakes are a shared problem. This is clear as day when you remember the “shared responsibility model”! In this model of cybersecurity, it’s commonly accepted that cloud service providers (CSPs) such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud are responsible for ensuring the cloud itself is secure, while organizations utilizing cloud infrastructure are responsible for securing what’s in the cloud. By nature, this emphasizes even further that ownership and responsibility are the name of the game, and as security mistakes can and will happen, no one person is ever alone or to blame.
Further, recognize that the systems that you work in aren’t perfect. The shared responsibility model between CSPs and their customers has gaps wider than the London Underground. As your whole team recognizes this, focus not on how each person might run around to fix as many problems they can—but on how you can use holistic programs and comprehensive platforms to maximize the benefit you can get out of every hour your team spends.
Leadership and Admitting to Oops
Every organization claims they want to create a culture where admitting to mistakes is allowed, it’s easier said than done. The best people to take the lead on this kind of culture are the leaders themselves. If you’re an organizational leader, you’ll need to nurture a culture where people expect that, when they come forward to admit to triggering an incident, you’ll be more interested in fixing it than in blaming them.
“When mistakes happen, don’t waste energy blaming people; invest energy in building better systems.” –Chapter 32, 1% Leadership
While you can tell people you’ll do that, you’ll also need to show them, repeatedly. You can prepare them. Talk about failing fast—which isn’t really a philosophy about failing more often, it’s a philosophy about detecting failures as early as possible, to limit their damage (which lets you move faster, which might mean you fail more often, but for less aggregate harm). That’s a core principle of various software management movements, from Agile to DevSecOps. Think about resilience engineering, which you’ll find covered in amazing depth by Kelly Shortridge and Aaron Rinehart in their new book, Security Chaos Engineering.
But don’t let problems linger. It’s important to take initiative for your team, whether you’re a software developer or a CISO. It’s inevitable that your organization will never be perfectly secure; new CVEs are discovered every hour of every day and it’s just part of the digital world we live in nowadays. Learn to be proactive and address gaps early and often, and know that there are a multitude of free resources available to help you along the way.
