The prioritization paradox
Every morning, the view from a security dashboard looks largely the same. It’s a wall of red text: hundreds of vulnerability findings, dozens of IAM anomalies, Terraform drifts, and a relentless stream of alerts from scanners, logs, and pipelines.
Most of these signals matter. Some of them matter a lot. And a select few are so urgent that engineering leadership should have known about them ten minutes ago.
But here is the gap: We don’t struggle with visibility anymore. We struggle with prioritization.
We have too much data and too little time to convert it into decisions. Right now, a true view of “What is our security posture today?” is buried across 15 different sources—from Trivy scans and Okta logs to CloudTrail access patterns and compliance checklists.
The result? Leadership doesn't get clarity, and engineering teams don’t get prioritized tasks. SREs like us spend too much time summarizing data instead of actually engineering solutions.
For a recent internal hackathon, we tried to solve this by asking a simple question: Can AI take all this noise and turn it into a small, crisp narrative?
The Concept: A Daily Security Brief
We didn't set out to build another dashboard. Nobody wants another dashboard.
We wanted to create a one-page narrative that an executive could read in 30 seconds while walking to a meeting, but that still gave engineers the evidence, raw signals, and ownership they need to fix the problems.
We built a prototype that combines our existing security signals, a lightweight logic-based scoring layer, and an AI summarizer (using RAG) to produce a "Daily Security Brief."
Here is exactly what the output looks like for an executive:
Daily Security Brief — 7 Nov 2025 Overall posture: Moderate (⬆ trending upward)
Top 3 actions
1. Revoke public access on S3 bucket customer-export-backup. Evidence shows external GET requests from unknown IPs. (Confidence: High)
2. Patch auth-api to address CVE-2024-XXXX. Public proof-of-concept available. (Confidence: High)
3. Investigate two unusual admin-console logins. Activity detected from a new geography/device. (Confidence: Medium)
Context & Narrative "Most findings today are routine, but two items elevate overall risk—a public S3 bucket containing sensitive exports, and a critical OpenSSL vulnerability with active exploit attempts reported. Combined with abnormal IAM admin activity, we recommend addressing these within the next 48 hours."
The "No-Magic" Architecture
To achieve this, we built a four-step pipeline that moves from raw data to a polished narrative. We were careful not to over-engineer this.
Step 1: Ingest and Normalize We aggregate outputs from all our scanners and logs (Trivy, Tenable, CloudTrail, etc.) and convert them into a single, normalized JSON schema. This ensures that a vulnerability in a container and a drift in Terraform speak the same language.
Step 2: Prioritize and Score (Without AI) We intentionally avoided using "black box" machine learning for the scoring layer. Instead, we used practical, logic-based scoring (0-100) based on risk factors.
- Public bucket + PII tag? High Risk.
- CVSS 9.0 + Internet-facing system? Critical.
- Drift on a critical Terraform module? Prioritize.
This filtering ensures the AI only focuses on items that actually matter.
Step 3: Retrieve Context (The RAG Layer) For the top high-risk items, we use Vector Search to retrieve context. We query our vector database for past similar incidents, relevant runbooks, PR diffs, and asset owner metadata. This provides the "memory" the system needs to understand if an anomaly is truly new or just a recurring false positive.
Step 4: AI Summarization with Guardrails Finally, we feed the top items and their retrieved context into an LLM. We use strict prompts to prevent hallucinations, instructing the model to: "Summarize the evidence below into 3–4 clear sentences for an executive audience. Do not invent any facts. Use only the evidence provided."
What AI actually does here
During this experiment, we discovered something surprising: AI is not necessarily great at "finding issues." Our scanners do that fine.
AI is great at compressing context.
The magic isn't in detection; it’s in summarization and narrative. AI helps answer:
- What changed recently?
- Which 3 things matter today?
- Who should take action?
This is the part humans waste hours on, and it's the part the AI handles beautifully.
Lessons from the Experiment
1. 70% of findings are noise
By forcing a strict prioritization model, we learned that the vast majority of daily findings don't matter right now. They aren't necessarily safe, but they aren't actionable today. AI helps identify the specific changes that require immediate attention.
2. Executives prefer narrative over metrics
Metrics do not equal understanding. Leadership prefers a narrative that explains "This matters because..." and "This impacts customers..." over a chart showing "500 High Vulnerabilities." The narrative format aligns Security, SRE, and Leadership instantly.
3. Engineers need the "Appendix"
While executives get the summary, the system generates a deep appendix for engineers containing raw scan results, CloudTrail logs, diffs, and linked runbooks. When engineers saw that the AI provided the evidence bundles and diffs automatically, they realized the tool wasn't replacing them—it was doing the grunt work they disliked.
The verdict
This system does not replace human judgment. AI cannot decide business risk, approve emergency patches, or understand organizational nuance. It acts as a "junior analyst" that writes the first draft and brings the right problems to the right people.
By applying practical AI with strong guardrails, we turned messy operational data into a system that saves hours of manual effort every week.
Conclusion
Modern security isn't a visibility problem; it’s a prioritization problem. We don’t need more dashboards—we need more clarity. By building an AI-powered pipeline to ingest signals and output a story, we can stop drowning in noise and start focusing on action.
