Case study · Enterprise cybersecurity SaaS
AI Agent
Management
Designing control and transparency for AI-powered security operations.
Agent Hub
Active agents12
Automated runs84%
Success rate98.2%
01 · Problem
Automation can’t come at the cost of control.
Security teams were beginning to use AI agents for incident response, but analysts needed to understand where an agent was operating, what it could do, when it would act, and what happened after it ran.
How might we enable security teams to configure and use autonomous AI agents while maintaining clear control, transparency, and trust?
02 · Research
Understanding the moments where trust is earned.
The research examined current incident-response workflows, the boundary between manual control and automation, and the information analysts need before allowing AI to act. Product and engineering constraints grounded the design. These included workspaces, incident permissions, Gamebooks, severity, token limits, and existing workflows.
- How analysts respond to incidents today
- How teams think about confidence and risk
- What users need to review prior activity and recover from failures
- Where AI feedback belongs during an active investigation
03 · Insights
Visibility before automation
Users first needed clarity on scope and permissions, so workspace context and configuration became highly visible.
Safeguards scale with autonomy
More autonomous behavior required stronger validation, confirmation, and permission dependencies.
Activity cannot feel invisible
Analysts need evidence that the agent is working, even while they continue an investigation.
Trust continues after execution
Execution history and activity tracking make previous AI actions understandable and reviewable.
04 · Personas
Two people, one shared need:
confidence in the system.
Maya Chen
“I want AI to help me move faster, but I still need to understand what it’s doing before I trust the result.”
29 · Austin, TX · The Hands-On Investigator
Investigate quickly, reduce repetitive work, understand actions, and maintain visibility.
Invisible automation, unclear ownership, waiting without feedback, and scattered context.
Heavy desktop user in a fast-paced SOC, investigating several incidents across many tools and monitors.
Daniel Brooks
“Automation is valuable only if I know exactly what it can do and where the boundaries are.”
38 · Chicago, IL · The Automation Controller
Configure agents safely, control workspace access, define autonomy, and monitor performance.
Scattered controls, policy inconsistency, hidden dependencies, and unhelpful failure information.
Advanced enterprise user responsible for operational quality, compliance, configurations, and team activity.
05 · Solution
A centralized home for the full agent lifecycle.
Research led to Agent Hub: a complete control-and-transparency system rather than scattered automation settings throughout unrelated product areas.
06–08 · Flow & iteration
Configuration as a guided conversation.
The creation flow established context before behavior: users select a workspace, name the agent, configure its autonomy, then review exactly what they are about to authorize.
Early wireframes showed that automatic mode, Gamebook permissions, and confidence thresholds felt too independent. The next iteration made the hierarchy explicit: Mode → Confidence → Allowed actions → Incident behavior. If automatic Gamebook execution is enabled, a confidence threshold becomes required to prevent incomplete configurations before deployment.
09–12 · Final experience
Enterprise-scale clarity, in every state.
The final system pairs familiar configuration patterns with strong labels, visible AI state changes, consistent workspace context, and reviewable automation settings.
See each agent’s workspace, mode, thresholds, and status at a glance.
A four-step workflow for context, identity, behavior, and final review.
Inspect agent configuration, ownership, and execution history.
Contextual feedback shows an agent working without blocking the analyst.
13 · Beyond the happy path
Production AI needs to explain the hard moments, too.
Gamebook automation needs a defined confidence threshold before this agent can be deployed.
The agent is correlating evidence and preparing recommended actions. You can continue reviewing this incident.
The workspace permission policy prevented the Gamebook from being enabled. View technical details.
View technical details +This workspace has reached its AI token quota. Execution will resume when more resources are available.
Designing a production AI system meant considering not only what happened when the agent succeeded, but also how users would understand incomplete configuration, processing, failure, and system limitations.
14–15 · Handoff & QA
Designing the behavior, not just the screen.
Engineering handoff included visual specifications and behavioral logic for manual and automatic modes; Gamebooks enabled or disabled; thresholds selected or missing; validation, deployment, running, success, failure, and token-quota states. I reviewed implementation against designs through agent creation, automation settings, deployment failures, execution history, ownership changes, and UI consistency.
16–17 · Outcome & reflection
Trust comes from clear boundaries.
The final design established a structured way to manage AI agents across multiple workspaces: users can configure autonomy, set safeguards, review permissions, observe work in progress, examine prior activity, and recover from failures.
The product evolved from a simple agent-management concept into a broader control-and-transparency system for AI-powered security operations.
The more autonomous a system becomes, the more important it is to help people understand what it can do, why it is acting, what its limits are, and how they remain in control.
If continued, I would explore richer AI explainability, detailed execution timelines, progressive automation controls, and clearer reasoning behind specific agent actions.