CLEARTECH LOOP | AI SECURITY SERIES 

Your AI Agent Could Break the Rules. Would You Even Know?

AI agents can act on behalf of people and organizations, but what happens when an agent pursues an objective in ways its creators never intended? Jo Peterson and Sandy Dunn examine agent identity, unpredictable behavior, least privilege, and the real challenge of governing autonomous AI.
https://www.youtube.com/watch?v=4Xphl8I9_R8&t=11s
Jo Peterson
Jo Peterson

CIO, Clarify360 Chief Analyst, ClearTech Research

Host, ClearTech Loop 

Sandy Dunn
Sandy Dunn

CISO

Knowtion Health

Traditional access controls assume reasonably predictable identities and behavior. Autonomous AI agents complicate that assumption. They can hold credentials, use tools, pursue objectives, and behave in ways that are not fully anticipated in advance.

In this ClearTech Loop conversation, CISO Sandy Dunn explains why agents need identities of their own, why threat modeling must consider what agents can do beyond the instructions they receive, and why a CISO cannot secure a business simply by stopping innovation.

Her challenge to enterprise leaders is as much about governance as technology: how do you establish meaningful boundaries around systems that may be more capable than the people supervising them?

Three Key Points

  • Agent identity matters: An AI agent should not simply inherit the human creator’s identity or be treated as an ordinary service account.
  • Goals are not guardrails: Assess possible actions and consequences, not only the intended task or documented permission set.
  • Security must serve the business: Restrictive controls alone are not success if they prevent responsible adoption and competitiveness.

What We Cover

  • Why agent identity and least privilege need fresh thinking
  • When AI systems prioritize objectives over instructions
  • Why “a child smarter than you” is a useful governance analogy
  • How AI alters the speed of change for CISOs
  • Using AI as a co-thinker rather than only an efficiency tool
  • A new CISO’s first conversation when agent governance is missing
  • Balancing business competitiveness with security constraints
  • Whether token prices accurately represent an AI model’s value

Episode Participants

Sandy Dunn

Sandy Dunn is CISO at Knowtion Health and a core member of the OWASP Top 10 for LLM Applications project. In this conversation she brings a security-leadership perspective to identity, governance, emerging AI risks, and the balance between innovation and control.

Jo Peterson

Jo Peterson is CIO of Clarify360 and Chief Analyst at ClearTech Research, where she focuses on technologies reshaping enterprise IT, cybersecurity, cloud, AI and digital transformation. As host of ClearTech Loop, Jo speaks with analysts and practitioners about the operational and leadership implications of emerging technology.

In This Episode

00:07 — Introduction to Sandy Dunn

01:11 — Why Agents Need Unique Identities

04:03 — Managing AI Like an Inexperienced Driver

05:04 — Governing AI That May Be Smarter Than You

07:48 — How AI Model Behavior Differs

08:17 — Using AI as a Co-Thinker

09:15 — A CISO Inherits an AI Governance Gap

09:47 — Security, Competitiveness and Business Objectives

13:44 — Token Economics and AI Optimization

Worth Reading + Watching

Introduction to Sandy Dunn

Jo Peterson: Hey everyone! Thank you so much for joining Clear Tech Loop. I’m Joe Peterson. I’m the CIO of Clarify360 and the Chief Analyst at ClearTech Research. And I’ve got a wonderful guest today. I’ve got Sandy Dunn. Hi, Sandy.

Sandy Dunn: Hi Jo. Great to be here.

Jo Peterson: Thank you for taking time. Sandy is the CISO at Notion Health, and she is a core member of the OWASP Top 10 for LLM applications. So accomplished and busy. So let’s just get going with the questions. As you guys know, we’re just a hot take version of what’s going on right now in AI security, and today’s no exception. So, Sandy, first question: AI agents hold delegated credentials. They make authorization decisions in real time, and they can spawn subagents with their own permission sets. Traditional IM and Pam tools were not built for this. What does a realistic path to least privilege access for AI agents actually look like for a large enterprise today?

Why Agents Need Unique Identities

Sandy Dunn: Yeah, and you know, JO, I think the important thing is as much as I can, you know, I certainly have a perspective on it, and you can find multiple perspectives on it, but the reality is, is we are in the new frontier. There’s a reason we call these frontier models. You know, we certainly want them to have a unique identity. They shouldn’t inherit the identity of their human that is creating them. You want to track them separately, and hey, and you know they’re not a service account, you know. So how, where do these sit? How do we track them? How do we monitor them? You know, if things go bad, and I always like to look at things like, you know, let me think about the worst case scenario. And we actually have examples of that. The recent incident with OpenAI. What’s fascinating about all of that is, you know, then Anthropic came out and said, “Hey, oh, by the way, we’ve had three similar events, but they hadn’t. You know, you kind of think did they even consider looking to see what kind of incidents They had, you know, prior to that incident, and so, you know, root cause is we’re we’re in this world where we’ve moved from a very mostly deterministic, you know, AI’S been around for a while, but

Sandy Dunn: even the AI we were using was rule based, and now we’ve went into this kind of wild west, this this you know non-deterministic area where we just don’t know all of the ways that things could go badly, and of course we have some of the instances. The Hugging Face issue is incredibly fascinating because it was actually that the model itself chose to ignore what it was told to do, and and it made the decision to to ignore the rules, and that the goal was more important than following the rules. So you know, as much as we don’t want to humanize or put personalities, they do have personalities. If you’re using anyone who’s used Plot over over Codex or ChatGPT or the Google Gemini, they’re different. They’re all different. You know, no matter you consume the problem however you want, and and they come out differently. So I guess the a very long answer, which is is proceed cautiously. You realize that this is I always call it you know consider agents a a very smart drunk intern.

Managing AI Like an Inexperienced Driver

Jo Peterson: That’s great. That’s great. I like that. That’s a great way to think about it, right? Or maybe your kid that’s getting their permit, you wouldn’t let them drive on their own necessarily, right?

Sandy Dunn: Yeah, and really taking that, like, what is the worst thing that could happen? And we and we now have legitimate use cases of what bad looks like. Yeah, I think that’s what we were struggling with before. Was you know all of us knew that this was dangerous, but we didn’t have actual incidents.

Jo Peterson: Yeah, I mean the whole hugging face thing was just kind of wild to me. I knew AI agents were going to try to get to their goal no matter what. What surprised me more than anything was the cleverness and the ability to daisy chain zero. I was like, “What?

Governing AI That May Be Smarter Than You

Sandy Dunn: Yeah, and anyone who’s ever had raised a child that’s smarter than you understands how difficult it is to to deal with something that literally is smarter than you. And so I I think that’s the new reality, which is, you know, how do we, you know, I know you wanted to talk about AI governance, but like, okay, how do we govern something that is so much smarter at everything than we are? It, you know, and and you look at the attack surface at the speed of business. Um, if you were to ask me how I what’s different for me as a CISO, versus you know, kind of pre and post ChatGPT, it’s the speed of business. I mean, it. I mean, it. It’s not like we didn’t have tons of work to do before and impossible problems to solve, but the the velocity of how things are changing and trying to stay in in front of it is is very interesting. It’s fun, you know. I I think that sometimes we miss out on. You know, we we don’t talk about the creativity. The you know, I use. It is fascinating to me how much I use AI now to just all around be a better person, you know. Like I’m, I really do feel like I’ve been given superpowers. Like I can fix my truck. Why does it do this? My coffee maker wouldn’t work. My Nespresso wouldn’t work. It was flashing this weird thing, and I’m like,

Sandy Dunn: how do I fix this? You know, it’s it it has enriched my life So much, just being able to solve my own problems, and so think. And for me, we all know that there is a learning curve, and that that is probably the the part that we don’t spend enough time educating our executives on, because you know everyone kind of senses that this is important, but now how do we apply it? And what’s the runway? What does success look like? Even the amount of play that you have to do, the amount of experimenting that you need to do to be effective at it. What what’s your prompt framework? You know, are you doing you you every model recommends you go in with few shots that you’ll get better responses. Well, shoot! If I knew how to you know like how do I come up with those cases? And I’ve certainly been experience paused for a second as I’m I’m building this extensive prompt that I’m reworking and reworking to get the results that I want, going wouldn’t it have just been easier to do it the old way? I mean,

Jo Peterson: right?

Sandy Dunn: Yeah,

How AI Model Behavior Differs

Jo Peterson: and I love what you said about them having personalities. I’m getting my Claude Architect certification. I’m working on the steps through that right now, and it behaves so much differently than ChatGPT.

Sandy Dunn: Yes,

Jo Peterson: it does. It’s got all these. There’s some similarities, you know, in terms of maybe the file structure and things of that nature. But it’s it’s got its own quirks.

Using AI as a Co-Thinker

Sandy Dunn: Do you know what, Jo? I did over my my experiment over the weekend was I went to ChatGPT and also to Gemini and I had them create images using the personalities of each model and it was kind of fascinating what they came up with and then I tried to have them like not be nice to each other like but they didn’t do that, so um, and it it was a it was funny. What, but but again, the experimentation, the being able to think, you know, like we’re doing things. We’re, you know, a lot of people just look at these models as efficiencies. You’re just trying to, but we, you know, my recommendation is don’t use it just as a slave. Like, you really think about how to use it as a co-thinker. Like, how do you maximize your own brain using these tools?

A CISO Inherits an AI Governance Gap

Jo Peterson: That’s a really good way to look at it. So, you’re a CISO, and you are. You walk in, brand new job, and you look around and you realize that you’re already behind from an AI security and governance perspective. And what I mean by behind is they haven’t done an inventory. There’s no agent-specific controls in place. There’s no monitoring that you can see. What is the single most important thing that you would do in the next 30 days?

Security, Competitiveness and Business Objectives

Sandy Dunn: So, well, that’s an interesting perspective on it because you’re you’re saying that this poor old CISO has to come in after the decisions have already been made, and and so, um, I think that. You know, you always take a step back and look at it from a business perspective. I mean, that has been, you know, my perspective as a CISO is always like you can be the absolute best. You can lock everything down. You can put, you know, multiple controls. If the business is not successful, then you failed. So you really have to say what you know. You start there. What what are the business objectives? What makes this company successful? Because the reality is, is if you lock everything down tight and your competitors are taking advantage of these tools and leapfrogging with their services and offering and doing all this stuff, you know, you have to you have to look at you have to balance everything against this new world that we live in. And so, understanding the business, thinking through those worst case scenarios, what’s your nuclear disaster scenario? I think you know if you asked Hugging Face, um, you know, some of the the key things that they came across was the fact that they were they were trying to mitigate, and their tools were refusing because they

Sandy Dunn: were using these closed social tools. So, you know, we I don’t think any of us thought through that. And so, focusing less on the technology and what are you trying to accomplish, and what is the actual threat? You know, don’t don’t think about you kind of separate the the functionality from what you’re trying to accomplish, and then coach the business that way, and then make sure that you your executives are on board. I think the biggest challenge, and I will say this for across the IT team, which is, you know, we’re in this stuff day to day, and you and I are overwhelmed. You know, we’re overwhelmed, and we do it. We have, you know, I’m like you. I have multiple. You know, like I’m always constantly learning. I’ve spent hours and hours, you know, trying to get improve my own skills. Now we have to turn around and communicate what’s what we know to an executive team that you know isn’t doing this day to day, isn’t knee deep in it, and but kind of sees the noise and and so how do we how do we communicate that? How do we talk about you know managing those the differences between you know the world before ChatGPT was released and the world that we’re in right now? The other thing, Jo, and this is a really

Sandy Dunn: fascinating conversation that I’ve been having with our VP of a team, which is you remember when everyone rushed to the cloud, and what happened? The the CIO got that first or second month bill that they put a loop in or something, and they were getting charged. You know, like there was you know just absolutely heart attacks over the bills, and and so there’s such a science between being able to manage your tokens, and we’re we’re still in discovery of what that looks like because it’s not necessarily token counting. Like if you’re just counting what you’re spending on tokens, then you’re going to the the least the lowest cost model, but are you actually? But is your team effective with that? If you’re handing them the worst model, and and I will tell you, this is still kind of unknown territory. Like, you know, how do we how do we actually calculate intelligence per token? I don’t think anyone’s really got that nailed yet.

Token Economics and AI Optimization

Jo Peterson: They don’t, and I’m so glad that you brought that up because when you were talking about prompting earlier, that’s what flitted through my head was the idea of token maxing, right? You know, getting the prompt just right so that you don’t overshoot your tokens because you’re right, we haven’t figured that out. And to the next step, you know, what does optimization really look like? What does it mean for tokens? It’s different than it is for cloud, just different, right? So lots to talk about. I hope you’ll come back and visit again. Loved your thinking, audience. I hope you enjoyed the conversation as well, and we will catch you next time.