Roey Rozi
Director of Solutions Architecture
Publish on
May 20, 2024
The transition to cloud computing has diminished the effectiveness of traditional network security perimeters, rendering identity as the final line of defense against attackers. While enterprises have allocated substantial budgets to identity programs and have made significant progress in safeguarding human identities through the enforcement of multi-factor authentication (MFA) and centralization of control via single sign-on (SSO), the broader attack surface presented by non-human identities remains unaddressed. In this presentation, we will delve into the security of non-human identities, exploring the various methods attackers can employ to exploit them in order to breach cloud-based organizations.
John Yeoh:
Hello and welcome to the Road to Cloud Security Alliance's AI Summit at RSAC. This webinar series is meticulously crafted to provide the most up-to-date tools and guidance on the critical topics shaping cloud computing today. While also serving as a precursor to CSA's AI Summit hosted on Monday, May 6th, 2024 at RSAC. Hosted by the Cloud Security Alliance, this series offers indispensable insights to aid organizations in navigating the intricate landscapes of cloud security. Join us every Wednesday and Friday throughout the month of April twenty-twenty-four. As we bring together renowned cybersecurity experts and community leaders to dissect crucial topics. These webinars promise to deliver key insights that participants can leverage to enhance cybersecurity within their organizations. Today we are joined by Roey Rozi from Oasis Security and John Yeoh from CSA to discuss breaking the identity perimeter. Now I'll pass it off to John to get us started.
Welcome everybody. My name is John Yeoh. I'm at CSA, and we are here, counting down the days to RSA where CSA has been pretty lucky. So we're counting down the days to RSA. We're kicking off our 15th CSA Summit. It's the 15th time that CSA has been able to really kick off the RSA event. So, that Monday, the CSA AI Summit is going to be taking place. And I'm here, counting down those days with Roey Rozi. He's one of the directors of architecture over at Oasis Security. And we're going to talk about just attack services. Attacks on-prem in new cloud environments. Identity seems to be such a foundation and a pillar of these things happening. But first Roey, I want you to say hi to everybody and just introduce yourself a little bit.
Roey Rozi:
Hello everybody. My name is Roey Rozi, director of architecture at Oasis. We're the leading non-human identity platform in this space and trying to protect everyone from this attack surface.
John Yeoh:
Indeed, indeed. Yeah. Super great to have you here. Man, our names go get together too, don't they? John Yeoh. Roey Rozi. I bet you a lot of people can't say that a bunch of times fast. But hey, we'll encourage them to meet us in person at RSA and you can talk to Yeoh and Rozi. So I think that'll be pretty cool.
Let's get into what we're going to talk about today a little bit here. I think we're going to have a pretty good conversation. The agenda slide here shows how we're going to just talk about today's attacks on on-prem networks. We're going to go into a little bit how cloud and the SaaS has changed everything. And then we're going to go to some actual examples too. So we're going to share examples of what we've seen in the industry as far as how certain areas of cloud are exposed. And of course, we want to give tips on how people can actually protect themselves against that too. So, Roey, I think we should just kick off and start talking about this. Tell me, how do we see ourselves and our organizations getting attacked these days?
Roey Rozi:
Okay, so I'll set up the stage a little bit on how attacks on on-Prem has been happening since 20 years ago and all the way until today. So it looks a bit like this. Your first login is usually one of two things. Either you have some servers that are internet facing, maybe through some type of a firewall. And you have to, in their internet facing some of them, you need to get a vulnerability to get a malware inside on the server. That's option one.
Option two is either a really sophisticated or a much simpler phishing campaign in order to get malware installed on one of the machines. Ideally some administrator or someone with enough privileges. And that's how you get your first login. Then you need to move in the on-prem. So there's two ways, you can either go with more vulnerabilities, moving inside, moving through the firewalls. Difficult, more vulnerabilities. Or, if you're lucky enough, you can find an identity which has more privileges and you can move around, get to file servers, get to other machines, until eventually you get that one really great credential.
A lot of time it's called the golden ticket. Usually in admin. Back in the day, it used to be just the admin human, but once we've got all of the PAM solutions, which protect the admin and rotate the passwords all the time, now it's all about those service accounts. You find that one old account that everyone's afraid to rotate because everyone is using it, and hopefully it had enough privileges. And if you found it, it's there for you for the next five years, for the next seven years, it's not going anywhere.
John Yeoh:
So, we always need a way in. And so we're looking at networks, we're looking at firewalls. We're trying to compromise an endpoint. And if there's an identity tied to that endpoint, that's what gives us that golden ticket you were talking about.
Roey Rozi:
And this is very common, and this is very hard by the way, to do these days. Because you've got so many different defense tools. You've got your... First of all, firewalls are great. They have AI, they have advanced capabilities. They're amazing now. And you've got a ton of detection both on endpoints and advanced detection systems that are capturing packets and working AI into them and getting everything, testing everything, checking everything, tons of detection layers. And also on the identity layer, you've got tons of tools for humans, you've got... Everyone has password rotation now. Every 30 days you're logging in, you need to put in a new password. And MFA is very common, and there is a lot of IGA tools where a human comes into an organization, a user is automatically created, he leaves, it's automatically disabled. So, a lot of this stuff is already pretty well protected, and it's becoming much, much harder to attack these on-prem networks.
John Yeoh:
Yeah. This is a good layer too. So we're talking on-Prem right now where you have these defensive perimeters, I guess you could say. And so from the network down to, we have control over devices that we can see, that we can hug, I guess, within our own on-prem environment. And so, things aren't always like that, I know, today.
Roey Rozi:
Yeah. But again, I mean, here, the situation is relatively all right. You've got a lot of tools. It's very mature. A lot of the people already know what they're doing. This hasn't changed a lot in the last 15, 20 years. So, we're doing pretty well. But what changed is now we're moving to the cloud and to SaaS. And I think every organization we work with pretty much has anywhere between 40 to 200 different SaaS providers, cloud providers, external identity providers.
And it's amazing for business because now everything is moving really quick and you can get tons of capabilities and everything is great for the business. But all of a sudden the network barrier is not there. And all that's left is that, and we'll show that in a second. All that's left is the identity parameter. And this creates a lot of leaks, a lot of hacks, a lot of exposures where an attacker is able to get just one identity that's leaked, stolen, off boarded employee that gave it to him, bought it off the dark web, phishing. And then you just need one of those, and you have access to a lot of assets which are not protected by the network parameter.
John Yeoh:
Yeah. I think you're being pretty conservative too with that estimate of 40 to a couple of hundred, depending on what you consider an asset too. I mean, we could be talking about thousands if we go into just... Not just SaaS, which we're using tons of. But, even inside how we're building containers, how we're looking at different types of just identities, period. When you move into the cloud, environment.
John Yeoh:
... just identities period, when you move into the cloud environment with services, with accounts, certainly with the people on my staff and all the devices they have. So yeah, it's pretty large.
Roey Rozi:
A hundred different staff providers, each of them could have hundreds or thousands of identity vendors.
John Yeoh:
Absolutely. Absolutely.
Roey Rozi:
What we've been seeing connecting to a Fortune 500 organization, in just the cloud infrastructure, they'll have around either a couple of tens of thousands of non-human agencies just in the cloud infrastructure. And that's a lot. Hard to protect, hard to govern.
So a bit on how attacking is now different. So you still have the classics, you can still do phishing, you could still do vulnerabilities, but now you have this whole area where all you need is one credential and you can attack it. You can get it through the phishing as well, or through dark web, stealing, leaking, mistakes. And then you can get, and there's no network parameter, these services are internet-facing by design. They're built to be interconnected and it's very, very difficult to create a network perimeter around them. There are a lot of organizations that are trying, but it's very, very difficult.
And then once you get one identity, for example, access to GitHub, maybe in the private GitHub repository, there's a different identity that connects you to the cloud, and then there's lateral movement inside of the cloud. You get one, you find another, you find another, and you're moving just through the identity perimeter. This is much easier for an attacker because you don't need much. You just need to start looking around.
John Yeoh:
Yeah, we certainly, we see advantages of moving to cloud too, where not only do you have the capabilities of the cloud services and the providers, but we do have some advantages when it comes to protecting those identities also, don't we?
Roey Rozi:
So I think something that's great is that a lot of the clouds have a much more modern identity infrastructure. I'll give you an example. So in Azure, in Active Directory, you have service accounts with passwords, and that's it. When you want to change a password, you have to reset it, put a new one in, and it's like a hot swap. So if you have seven services using that password, it's very hard to do the rotation.
With a lot of cloud and SaaS providers, they solve this issue and you're able to create multiple secrets. So it's much more modern. You have a lot of modern infrastructure that helps you, provides you more granularity and more control, but it's also harder to manage. So you need the know-how and the right tooling.
John Yeoh:
So we're focusing on slightly different things once we move to cloud. And I think that's a good thing to note where we have a lot of new capabilities when it comes to identity, but new challenges are introduced.
Roey Rozi:
And I think this is a great opportunity for a lot of identity teams that have been working a lot in the on-prem to move over to the cloud. Because what we've been seeing is that a lot of organizations have sort of clash between identity teams and cloud teams, and they all want to have things secure, but who's responsible for the identities in the cloud? And that's an interesting question, which I see different organizations tackling differently.
John Yeoh:
Yeah. I'll echo that. I'll echo that for sure too, where our identity teams and our cloud teams communicating with each other, do they have the tools to do that too? That's always an important thing.
Roey Rozi:
I think what's usually happening is that each one will have its own tool and they're separated. They're fighting on resources. And in most of the organizations I've seen, it hasn't been going smooth.
But at the end of the day, what happened? Because you have to use cloud for business. I don't think there's any other way these days. And so more and more assets are now in this golden rectangle and they're there and you have to find a way to protect them. You just have to, because the business is moving there.
John Yeoh:
Yeah, so we have all these new API vulnerabilities introduced as well as just that scale of accounts that you mentioned before, no? Hundreds of thousands of SaaS accounts that we're using, or I should say SaaS products that we're using with another thousand accounts per... Yeah, that's a pretty exponential issue right there for sure.
Roey Rozi:
Very hard to control. And talk a bit later about how you can do it anyway, but definitely requires to know how to do it in a lot of tooling as well. Either do it yourself or has to be created by someone.
Great. So I think something I mentioned, I'll go over this just 20 seconds. I mentioned non-human identities or NHIs. And what I mean by that is anything that's being used to connect between machines or that's not tied to a specific human, and there's so many different types, and you can Google this later and see the different definitions and it's complicated. But just think of it as anything that can let you connect that's not tied to a human. And when he leaves the organization, it's not automatically the same.
John Yeoh:
Yeah. So not just machines, right, we're talking accounts. I think that's the biggest thing to note there too.
Roey Rozi:
Even think of a service account that someone created, but it's not tied to his name and it's being used by some script on his computer. If it leaks, someone can use it. But when he leaves the organization, it's not deleted. It's there forever.
John Yeoh:
Yeah. Okay.
Roey Rozi:
And as I said before, attackers love these, love the NHIs, because they don't have any MFA, they're very difficult to manage. It's hard to rotate because you don't know who owns them, you don't know what services are using them. They're very rarely deprovisioned once no longer in use. So you'll have a project, you'll create some identity for it, and they'll be there forever, usually not removed.
And another great use for those is backdoors. So you get some type of access to an organization, maybe temporary, maybe just for a couple of hours, but if you have enough privileges, you can create a new non-human identity and no one will know. It'll stay there forever. It's very hard to audit.
John Yeoh:
And it's interesting too, because I think a lot of people always think that MFA is the answer, but you can't exactly implement MFA on these type of non-human identities. And we know that rotation of these service accounts can be super difficult and challenging to do with just the scale. We talked about how many accounts there are, so rotating those keys and rotating those accounts can be a little nightmarish, even a small company like us.
Roey Rozi:
You're afraid. You're really afraid. You don't want to hurt the business. Security can never hurt the business.
John Yeoh:
Yeah, we should patent that.
Roey Rozi:
Well, that's what we try to do.
John Yeoh:
Great. So tell me more about what we're looking at here with this architecture. Is this just, we're just capturing how NHIs fit into that previous architecture, right?
Roey Rozi:
Yeah. So this is just, we've got the standard on-prem. This is a simplification. It can be a lot more complex. And these are all of the externally facing services. These are services that are not protected by a firewall. All you need is an identity in order to access them.
And what we've noticed that a lot more assets are now in this space. You've got Office 365, and you can just go online and connect to some SaaS tool that will connect to them directly. And it's great. You can do a lot of things. You can do automation, you can hook up ChatGPT to use Office with other AI tool. You can do a lot of really cool things. But it means that everything is internet-facing by design. So you have to be a lot more responsible [inaudible 00:15:47] now.
John Yeoh:
And we've seen this, we've seen some of these NHIs getting exposed and compromised.
Roey Rozi:
Oh, yeah. I'll show you an example. So this is an example from Microsoft published a few months ago, and what's happened here is this is-
Roey Rozi:
... a few months ago. And what happened here is ... This is a simple, warm-up example. Really, really simple. The Microsoft AI team released some kind of a public repository for the public and they wanted to share some files, so they created something called a SAS token. This is a type of a non-human identity for Azure Storage accounts. And they configured it to expire only 2051, which is not the end of the world because they wanted this publicly-facing, but it shows you the low level of control. And the worst thing is they accidentally gave more access than they wanted to. They wanted to give access only to a specific container with some source code, but accidentally they gave access to the entire Storage account, which much more files.
And the reason this happened is because there's no guardrails. When you're creating a SAS token it's very hard to control, and there's no tracking, no monitoring. There's no audit on them, by design. I'll show you. Let's create a SAS token and let's see how it looks like. Let's see. This is a Storage account, my Storage account, you can see, in Azure. And you can use regular RBAC to access it, and that's the recommended way. So you can click here and there's regular RBAC. Classic Microsoft, fantastic. You also got access keys, which can let you access the storage account. You just click here, Reveal, and it'll show you the access key, and you can use this to access the Storage account from anywhere.
But if you want to share access with someone, you can use these keys to sign what I called SAS tokens or Shared Access Signatures. And it looks a bit like this. You just choose the service. I want to give access to these services. All the files and blogs and containers in the Storage account. Default expiry date. You can put here, let's say ... I don't want it to expire right now, so I'll put here much later. Developers are lazy, this is what they're going to do. And that's a generated one. Can see it here. I'll rotate this later. Nothing in the Storage account, but that's it. And there's no tracking, no auditing, no monitoring. It's meant to be easy and useful, but it's hard to control.
John Yeoh:
Yeah. And it's so important to note too that these were good intentions when this was shared too. I know it's the Microsoft AI team that was sharing Open Source code for everybody to use, but there was just some excess files that were shared. And that was the issue that was exposed to you, right?
Roey Rozi:
Yeah. And I think this is just the problem because when you're creating such tooling, you have to provide a way to audit it and control it and monitor it. And that's what we're trying to build.
This is a screenshot from our product, from Oasis. It's just next to all of the audit logs, optional audit logs, turn on everything, and it's able to build a view like this showing the access key and the SAS token and who's using it. And you can do this yourself. It's relatively complicated, you have to track all the audit logs, and I don't think someone can just do this.
John Yeoh:
So in this here, we're trying to share something in Azure with good intentions. We even have the SAS, SAS tokens, which are supposed to help limit access. And so we're implementing privilege access, which is a good thing. But we need something that's going to catch us in case we make a mistake there because we're still looking at humans creating access tokens.
Roey Rozi:
Yeah. For example, with human identities, you're required usually by compliance regulations to every six months do an access review. Look what's connecting, who's using what, is it okay, is it not okay. You don't even have a view like this built in for a lot of NHIs, in [inaudible 00:20:04].
Roey Rozi:
But that's better than nothing. And it's even hard to revoke them in this case. So the SAS tokens are signed by the access key, and the only way to revoke them is by rotating the access key, which destroys all of the created SAS tokens. So if you have multiple, there's no way to revoke one. It's difficult.
John Yeoh:
Yeah, absolutely. That is a challenge.
Roey Rozi:
All right, so that's just one sort of warmup example. And I think this is going to become much bigger. I have a term which I like to use, which I call attacking for infrastructure. This is a different flavor of supply chain attacks, but I look at it from a different perspective, from the attacker's perspective.
So attackers are a business, they're doing operations for business purposes. They want to hack networks, get data, get money from it, for different reasons, different ways. But at the end of the day, it's a business, it's an operation, and they want to be efficient so they're creating infrastructure for that. And the infrastructure has many different types. It can be automation, it can be tooling that helps them be more efficient, but it can also be other operations.
What I think is happening now is that a lot of groups that are well-funded and organized are doing operations that their whole purpose is to make and simplify multiple future operations. So the smaller players, what they'll do is they can hack small fast tools that have intimate access. Think migration tools, analytics tools, security tools, management tools, smaller players. And big well-funded attackers can be more sophisticated. They can go for the big players, the identity provider, the cloud providers. And if they hack Okta, they can now access maybe hundreds or thousands of Okta's customers and get a greater result from their attack of infrastructure.
John Yeoh:
Yeah. So we're looking at that's the game you were talking about here too, where there's different levels of attacks. And if we have larger scale attacks that can go after some of these providers with these identity tokens and authentication tokens, that's where you can get that golden ticket you were talking about. Now this is an even bigger golden ticket than the on-prem stuff you were talking about, right, because now you have exposure to multiple organizations.
Roey Rozi:
Yeah, This is the foot in the door to every organization, or thousands of organizations. Think, if you have intimate access into Microsoft infrastructure or into any other cloud provider, any other identity provider that is connected into many, many different organizations, this is the foot in the door that you're now getting for everyone at the same time. Very efficient.
John Yeoh:
Yeah. And really, I know where people will say like, "Hey, you guys are cybersecurity professionals. This is what you do. You think of worst-case scenario or you're looking for exploits everywhere," which is, yeah, that's what we do as researchers too, but we have examples of this in the real world too, right?
Roey Rozi:
Yeah. I think this is happening all the time, and usually we're not even hearing about it, but recently we heard about it, so that's great. Well, not great that it happened, but great that we heard about it.
I'll tell you the story of what happened with the Okta and Cloudflare breach. So an attacker, probably relatively sophisticated, was able to hack into a personal computer of an Okta help desk employee. I don't know how, phishing, something creative. And then from there, the Okta help desk, he worked with a lot of different admins, and whenever there was a problem he requested if they can please send him an HAR-
Roey Rozi:
He requested, if they can please send him an HAR recording, a network recording, from their browser, which makes sense. They're Okta help desk employees, they can help him log into Okta and the attackers captured those systematically for a while, and then they can use those short-term credentials to hack into customers' Okta. Specifically, they hacked into Cloudflare's Okta. And when they're there with enough credentials, they can get all the secrets of all the non-human identities. So they can connect into everything.
Now, what happened here is that the Cloudflare team was really, really proactive, and this is super impressive. And the moment they heard about the Okta breach, they went around and rotated every single identity, and I have no idea how hard this was. They were there for weeks, probably. Working every day, every night. And they missed four. They missed four identities, which they didn't rotate. And the attackers were then able to leverage those four identities to get into other assets to move laterally into Cloudflare. And luckily, a detection system caught them, and they were able to then rotate all of the identities again. This time they didn't miss any, and hopefully everything is now good. But this is when they got caught. I'm guessing these types of attacks are happening all the time, and we don't even hear about them because it's very efficient. You have to hack one help desk employee, and this can get you access to hundreds of organizations.
John Yeoh:
It's similar to what you shared earlier too, with the on-prem stuff, but now we're looking at that grand scale of things, right? You're looking for a way in, you're looking to deploy. Whether it's malware or snooping around, or looking for certain credentials and whatever golden ticket that you can find, and then exploiting that. Roey, do we know why only four identities were left unrotated, or just that weren't rotated at all, or was that just an oversight?
Roey Rozi:
I think it's just an oversight. It's so hard to get everything.
John Yeoh:
Yeah, it certainly is. We talked about just the sheer amount of accounts that we're talking about and identities that we're talking about, so yeah. Yeah. Pretty crazy.
Roey Rozi:
The way I'm imagining it is a team of 10 people, 10 engineers, with giant spreadsheets, shared spreadsheets, marking them in yellow and orange and red and green, and like, "No, I did this one. I didn't do this one." "Are you sure you did it?" "Yeah. I'm sure." That type of hectic.
John Yeoh:
Yeah, and I think we see these breaches or these attacks or whatever we want to call it. Yeah, they're definitely attacks. When we see people react and respond to them, they tend to do a pretty good job, but it's still super resource intensive, and you just explained not just the amount of workforce it took to rotate those service accounts, but it's a stressful, panicked environment for those employees too. So having better visibility or just have a better understanding of what's at stake is super important. And having the good partnerships in your cloud ecosystem is just as important too.
Roey Rozi:
And I think this is just going to grow. So now they only had to rotate 5,000, but think advances in AI and automation tools and third parties, and this will be 50,000 next year, or not next year, maybe five years, it'll be 50,000.
John Yeoh:
Oh, yeah, maybe five months from now. I mean, maybe not even a year or five years from now, the way things are going. But yeah, I think it's important to talk about some of the theoretical stuff that you and I were sharing earlier, but also just, "Hey, what are the real world examples that make this real?" It's happening. It's happened, and we can learn from them, one, but also we can make sure that we're implementing the right preventative and even detection measures.
Roey Rozi:
What I would recommend is that every organization that's serious about security will create a plan on how they can rotate all identities within a week or within a few days with a minimal disruption of business and create this plan, maybe even practice it, understand what tooling you need, what processes, because this will happen eventually, maybe in a year from now or five years from now, but you'll have to use that plan when there will be a breach similar to this.
John Yeoh:
I think it's a great exercise to do, just a good tabletop exercise for your response teams and for your... We talked about just, "Hey, how do identity teams talk to your cloud teams and talk to the SOC and NIR teams too?" So, big flow of communication, a big understanding how to do that. Yeah, I like it. Practice makes perfect, right?
Roey Rozi:
Yeah.
John Yeoh:
Yeah.
Roey Rozi:
I really recommend it.
John Yeoh:
Yeah. Good stuff. Good deal. All right. Roey, maybe close this down here real quick too. I think this is a great example, like I said, of just understanding today's modern environment, how we've grown from modern or from on-prem to these cloud environments. And I think it's silly to think that any of us are not in a cloud environment and not exposed, or you don't have these kinds of identities and accounts that are a part of your environment that you need to be paying attention to. So, summarize what we talked about today for us. Will you?
Roey Rozi:
Yeah, so I'll summarize it the way I see it from the identity perspective. So identities in the past, in the on-prem, were mainly used for lateral movement. So you should worry about them, but they aren't the first thing you worry about. But with the movement of all of the business assets into cloud, now identities, they're the perimeter. They're the main defender for your assets. So the main takeaway I would recommend from this is that you start thinking of them as such a perimeter. Take care of them. And, out of them, human identities are relatively solved. You've got your MFA, your IGA, your PAM solutions. Things are relatively all right, you can always improve, but there is a lot of existing tooling and know-how and how to do it. But with NHIs, with service accounts and tokens and API keys, I think a lot of practitioners are still either unaware or scared to try and tackle this problem. And I think we got to the point where we have to, the community, understand how to tackle this issue.
John Yeoh:
And that's perfectly said too. If you're looking to protect your data, that almost always starts and ends with identity and access. And we need to recognize not just the humans, but especially the non-human identities that are involved in these modern IT systems, modern organizations, that we're going to have a lot of those. So fantastic stuff, and we'll talk about this and more at RSA, certainly at the CSA AI Summit that Monday. So make sure everybody comes early so that you can... Hey, Roey, come chat with us on Monday, Tuesday, Wednesday, Thursday. I'll be there all week. How about you?
Roey Rozi:
I'll be there as well.
John Yeoh:
All right. Can't wait to see you in person. Thanks everybody for listening to us. This is our Road to RSA, Road to the CSA AI Summit. Looking forward to seeing everybody. Roey Rozi, thanks for sharing.
Roey Rozi:
Thanks a lot for having-
