Hyperscale Security

To document all of the weird things we run into in cloud security, I long thought someone should be the "rachelbythebay" for secure cloud transformation...

It is a common refrain that security teams are under-resourced and under-staffed, and that they are not supported by the business. If that is truly the case, you're probably in the wrong place. But what often happens – and is something we at least have control over – is that we fail to communicate in a language that the business understands.

Talking To Ourselves

Many of us in #infosec come from technical background, and many get an introduction to our culture from hacker and cybersecurity conferences. The talks there are often highly technical, and even if more abstract, filled with jargon unique to the field. Vendors and analyst firms add an additional layer to this, that is hard to follow even if you are in the industry.

You are probably not going to get heard if you ask for funding to implement protections against the OWASP Top 10, deploy a WAF/IPS, cycle vulnerabilities with HIGH and CRITICAL CVE ratings, implement MFA and embark on a multi-year Zero Trust Architecture deployment.

Talk In Terms of Business Risk

Senior executives go from crisis to crisis and only what is most urgent will occupy their attention. They manage all kinds of business risks, of which security may not be on their radar at all.

But they do understand probability of risks and financial impact. Your organization almost certainly already has established scales for probability and financial impact thresholds that can be plotted on a 4x4 or 5x5 grid to articulate business risks. When security risks are communicated similar to how business risks are communicated, it is much easier for executives to compare where they sit relatively to other risks they need to manage.

Learn How to Prepare an Executive Deck

It seems only rarely anybody teaches you how to prepare a proper executive deck, and you tend to learn by participating in the process. As this process can easily lead to multiple editors changing language to the point of meaninglessness, the more you can get your deck into the expected shape, the less likely it is that people without domain knowledge will make changes before it is presented.

  • Everything you want to say must fit in 3 slides maximum
  • Make sure all that is most important to you is in the first slide — there is a realistic possibility you may not get beyond the first slide
  • Realize that many execs are high-speed information absorbers, so get to the point immediately. Make your point in 5-10 minutes

Prepare a Realistic Plan, including Budget and Number of Resources

The last thing you want is to succeed in getting attention and support, and have no answer when you are asked what you need to bring down the risk. That is your opportunity wasted. Make sure you have a budget and resource plan ready that shows what you are going to do, justifies the budget and articulates how this reduces business risks. Include clear milestones, dates and meaningful progress metrics that can be reported on.

Good communication typically comes from placing yourself into the position of others, and speak to them in language they understand. A good start there is to read any corporate strategy documents you can find, and align your message with its main messaging and how security supports that. You are bound to get a lot more traction that way.

cloud security posts without corporate approval @jaythvv@infosec.exchange

Recently, I sat through a presentation by a network-provider-turning-into-a-cybersecurity-vendor that showed a diagram of IaaS, PaaS and SaaS and the need to secure that full stack. I agree in principle.

Apparently, though, the big missing piece in that was Network Security as a Service.


There are many challenges I have been involved in working through in cloud security the past years. But that this was the big gap never occurred to me.

The Death of The Perimeter

The combination of the migration to public cloud and working from home/anywhere accelerated by the global pandemic has led to many people to declare the death of the network perimeter. Instead of resources being safely (...) protected behind firewalls in a trusted network, suddenly we have a multiplication of end points and network connections.

This is almost invariably seen as a big problem that adds to all the challenges we already face. An alphabet soup of solution categories are offered to get around this and everybody is talking about Zero Trust Architecture. I am not disputing these are good things, but they all require a lot of effort and money to implement and operationalize fully. It seems we don't even consider that there might be benefits to the perimeter's demise.

Free Micro Segmentation

And there certainly are. Each cloud account (subscription or project, depending on your platform) is an isolated virtual data center. Aside from wide-open default VPCs (which you should nuke or neutralize immediately), there is no network path at all between that cloud account, any other, or the outside world. Each has to be configured individually, or network segments opened. Voilà, free network micro segmentation and reduction of blast radius.

Out-of-band Administration

Whether you are using the cloud web admin console or, preferably, the cloud API and Infrastructure-as-Code, you are communicating with the cloud provider's API, not via a direct network connection to the landscape inside the cloud account. This takes place entirely out-of-band from a network perspective, and may as well be an air-gap.

If you really, positively, have to have SSH access to your landscape, you can easily configure the environment to only accept network connections from selected CIDR ranges.

Identity and Authorizations

With all due respect to everybody in networking, we absolutely appreciate everything you do to ensure that networks exist – but we less and less rely on network security to keep landscapes safe. That is, we essentially treat networks as inherently hostile. Network traffic should be encrypted end-to-end to a limited set of ports. Everything between network boundaries is an API communicating over 443.

The main security control is Identity and Access Management, or Cloud Infrastructure Entitlement Management (CIEM) if you like. Who (or what service) you are, how you authenticate yourself, and what authorizations you have is the main protection. What network path the request took is increasingly irrelevant while the network traffic is encrypted anyway.

Cloud Mindset

We shouldn't build controls for a new age based on the challenges and solutions of old. Building Maginot lines of lifted-and-shifted security solutions, patched up with new tools that try to recreate the safety of old is a fool's errand. It presupposes our “trusted” networks were safe in the first place.

We have to understand how the cloud work to properly protect it against the risks we face. This allows us to see new paths, and the power it provides as well. Policies applied at organizational level on all cloud accounts in the organization provide a level of enforced controls that are nearly impossible in the data center, without really costing anything. Central logging simplify how you monitor network flows. It's not all bad. Much is much, much easier.

When you hear us cloud hoodies go off on our mythical cloud-native mindset rants, this is the sort of stuff we mean.

cloud security posts without corporate approval @jaythvv@infosec.exchange

Cloud Security is constant crisis management. Not in terms of panic, everything is on fire, but as a constant mode of being. Should a fire breaks out at your house, I bet you'd be in panic mode, as I sure would be. But for the Fire Department it is just another Tuesday. (literally)

I was reminded of that when I saw two separate reports of vulnerabilities of the cloud come out just today:

Told You So Doesn't Stop Us From Having To Deal With The Impact

These are great write ups with technical details, and you should read them. But since they're already fixed, they are largely like a weather report for a different continent.

Hurricanes or tornados way over there may make the news but don't affect you as much as the showers or snow you may be dealing with back home[^1]. Scientists and activists have warned us correctly for a long time about climate change and made recommendations that for reasons outside their control weren't followed. First responders nevertheless have to manage the local disasters where they occur and do what they can with the resources they have available to them.

Security professionals similarly have offered good advice for decades. Also here often lack of resources, commitment or willingness to face consequences meant their recommendations weren't followed. Yet those in the trenches try to make systems secure and respond to alerts to the best of their ability, while security operations deals with the fall-out with the capabilities they have.

Managing Cloud Turbulence

Cloud transformation only adds to this sense of crisis. We rush head-first into cloud platforms and new cloud-native technologies like Kubernetes, little of it secure-by-default. We give more autonomy to developers, while we're still figuring out how the secure these landscapes. Changes to the way we work, develop and deploy code, and adjust organizationally to cloud transformation only add to the turbulence.

We only manage this by realizing we are in a state of constant crisis and will continuously have to make difficult choices. This is what crisis management is for. When you rarely have an incident, you may have good playbooks, but when an incident happens nobody knows what to do. When incidents are routine, you get a rhythm and comfort level with them. You manage.

That doesn't make me less concerned about these vulnerabilities – we rely on the cloud provider for the security of the cloud in our own threat modeling and processes. I am concerned about the impact of climate change on communities around the world, as well.

However, we all first have to deal with the crisis in front of us and prioritize according to the impact on our organization for the things we have control over and can mitigate against. It won't do to stand on the side lines and say “I told you so”. We have to save what we can. Welcome to constant crisis management.

[^1]: I am in the greater Santa Cruz area, recently hammered by massive rain storms. While our little community is OK, thankfully, many areas have seen major flood damage and the risk of landslides continues. Extreme weather events are obviously on my mind.

cloud security posts without corporate approval @jaythvv@infosec.exchange

How does anyone think this works?

Sponsored/InMail: Hi, my name is XYZ, Co-Founder and CEO at ABC, dedicated to solving the big problems in cybersecurity. Here's a link to my calendar so you can schedule a 30 minutes demo

Cybersecurity vendors, please stop doing this. I get a couple of these per day on LinkedIn or email, and the only thing it does is get you remembered for all the wrong reasons. Cold contacts like this literally scream that you have no idea what you're doing.

We all have a lot of work to do, and if you are in the cybersecurity industry, I assume you are in it to make the world more secure. Therefore, allow me to share some insight from the other end, and hopefully you can spend your energy better.

We Study the Market

Large organizations with a significant security organization follow what is going on. Different teams will track different segments through the industry media, analyst briefings and reports, and what we hear our peers are using and their experience. We attend conferences, watch webinars and listen to podcasts. We get regular dedicated presentations from established strategic partners. We have partner teams and investment arms that survey what the cybersecurity industry is doing.

You don't have to contact us – certainly not over LinkedIn sponsored direct message! – we will contact you. For all of our new cloud security tooling, we reached out to the vendor. Two of them we talked to for use cases we identified them for that they weren't even selling the product on. One of them, we contacted when they just came out of stealth, after an analyst had talked them up during a briefing and got us interested.

If we don't contact you, it's because you are not a realistic candidate. (Sorry)

Do Your Research

During my consulting days, before every job, I looked up their website and searched around to get a sense of what the organization was doing. If you give me in the first contact the idea you don't know what we do, how big we are, or what challenges we may face, you have just shown me you wouldn't even do that.

There are ~3,000 people in our organization involved in security in some form or fashion. We operate cloud services with critical workloads for paying customers. Established vendors have seen their products fail in our landscape. We have spoken on security and DevOps conferences about the challenges we've faced that are an easy search away.

Are you sure you would even want us as a customer? Are you ready?

I Get How This Would Be Good For You, How Is It Good For Me?

I have been in the tech industry since the 90s. I totally get why it would be a fantastic opportunity for you to have a customer this size and name. The marketing potential alone would be excellent, wouldn't it?

Cool. But what is the problem you are solving for us, though? What remaining gap do you cover? Why would we replace an established vendor for you?

At Least Give Me A Reason To Be Interested

The effort may not be entirely wasted. It's always possible that you are doing something very cool and innovative. It is useful to have a product sheet and a tech paper that allows me to place you in a particular segment. That still likely won't lead to a follow-up, but if it is interesting, I will remember you. If I hear others mention you, I will ask them about you.

There is at least one company, though, that keeps contacting me and never seems able to explain what they do. That is all I remember about that company. I talk to others about them and their sales approach. Don't be that company...

But My Product Is Different!

You may genuinely believe you have a solution I should know about and your startup is different from the pack. It may well be! So, tell the world, get onto conference talks and podcasts, and release technical papers. Get the attention of the analyst firms.

But realize that a company our size has 18-24 month procurement cycles with multiple layers of approval. You'd get fully scrutinized for multi-year viability, likelihood of being sold to a larger company, level of funding, and all that for the duration of a multi-year contract. You would have to prove why you're better than a preferred vendor that has more resources than you. You could run out of runway chasing a whale, when you could be building the business fishing for cod.

If you're good and get traction, and solve problems for your customers, we will notice you. We will follow your progress. And if we think you could fit in our plans, we will give you a call.

Just writing this, I already thought of enough material for a future follow-up: how not to screw up your first meeting. Stay tuned!

cloud security posts without corporate approval @jaythvv@infosec.exchange

About a week ago, I was in a quarterly security review meeting with one of the business units, including the relative business leader and BISO. All of us have been in the tech industry for decades. We discussed the progress of the last quarter in the middle of rapid cloud migration. As an aside, we agreed the three of us that cloud transformation is the hardest thing we've done in 30 years.

Over the last 4 years I was deeply involved in cloud security during a rapid cloud transformation that is ongoing (arguably, it never ends). As cloud transformation reaches into business functions and processes you never foresaw and poses challenges you never thought you would have, it is a journey filled with absurd and crazy adventures.

Hyperscale Security

I ran a company-internal blog to document what we were going through as a running commentary, simply because I felt some sort of record needed to be kept. Increasingly, I thought those struggling through their own cloud transformation could benefit from such experiences. I have always liked the 'war story' posts on http://rachelbythebay.com/w/ and thought I might do the same. I even got the domain name – hyperscalesecurity.com – but for months hadn't figured out what platform or how to host...

Jerry from Infosec.Exchange to the rescue! Long live the Fediverse.

cloud security posts without corporate approval @jaythvv@infosec.exchange