Threat-modeled AI agents in live operations

The problem

Live operations is full of toil that looks automatable and mostly isn't being automated, because the people who can build the automation have never carried the pager, and the people carrying the pager don't ship tools. There is a second, quieter problem: most AI automation gets built with no security review at all, which is how an agent with mailbox access becomes an incident of its own.

The method

I built AI-assisted agents for the Xbox Operations Center, and I put every one of them through Microsoft's Responsible AI review with a threat model before it saw production. The threat modeling was not paperwork theater. An agent that reads email and acts on it is an attack surface, and treating it that way from day one is what let these ship at all.

Two examples:

• The maintenance-suppression agent. It reads incoming maintenance notification emails, parses the window with an LLM, and suppresses alert paging for the affected services during that window. On-call engineers stop getting woken up for maintenance somebody scheduled three weeks ago.
• The outage-communications chain. A full skill-chain built on Microsoft's Copilot stack, Copilot Studio agents plus agentic Copilot CLI workflows. During an incident it reads the ticket, assesses impact across services, links related tickets in partner queues, identifies which team to engage, and drafts every outward communication: customer-facing status, social posts, and the internal comms fields. It saved roughly fifteen minutes per communication.

The part of the comms chain I care most about is the drift measurement. The chain diffs what humans actually sent against what it drafted, which turns "do we trust the automation yet" from a debate into a number. Shrinking drift meant the tool had earned the next piece of the job. Stubborn drift meant a human still owned that piece, and we could see exactly which piece. That is how I think automation decisions should be made in operations: measured against real output, not against a demo.

The labeling matters to me: this is AI-assisted work, not solo expert programming. My contribution is knowing which toil is worth automating, what the failure modes look like from the operator's side, and how to get security sign-off so the thing can actually ship.

The outcome

Real toil left the on-call rotation, with a security review trail behind every agent. A colleague's written recommendation credits me with leading the team's AI adoption, which I mention because third-party confirmation beats self-report. The combination this work represents, practical AI automation plus operational judgment plus a formal security process, is the thing I would bring to any team modernizing its operations.