Tuning Detections isn’t Hard Unless You Make it Hard
Cybersec Café #71 - 06/10/25
Detection Engineering is an underappreciated role in cybersecurity.
Is it the flashiest? The most technical? The most revenue-generating role in tech? Definitely not.
But it is niche, highly valuable, and a revenue saving role when done well.
At its core, Detection Engineering is about writing and refining the rules meant to detect threats in your environment. It gives the security team complete and customizable visibility into the tooling deployed across your environment.
And depending on how many platforms and tools you plan on monitoring, this job can quickly go from manageable to overwhelming.
Just think about it: You purchase a new application. You onboard the new log source. Your SIEM has no out-of-the-box detections for it. You spend a couple weeks parsing docs while the logs start trickling in. You write some detections to start gaining coverage, just to have low confidence they’re valuable. And then…
Boom. Your team gets flooded with alerts. All false positives. All adding noise to an already overwhelmed SOC. Now your team is frustrated and you’re starting to feel the pressure.
This is where tuning becomes critical.
It’s unrealistic to expect your first iteration of a detection to be perfect. Sometimes there’s just not enough context or data upfront to have high confidence from the start.
The goal is simple: We want detections that identify real, meaningful threats - not rules that drown your feed with noise.
Here’s how to tune your detection suite at scale with clarity and confidence.
Detection Tuning
In order to build a process that scales, you first need to understand the most common ways to improve your detections.
These aren’t one-and-done tasks. Tuning is iterative. You’ll often need to apply multiple techniques in combination to tailor a detection to the unique characteristics of your own environment.
Logic Adjustments
Logic adjustments are the bread and butter of detection tuning. They come into play when your detection isn’t quite aligned with the real-world behavior it’s meant to catch.
Sometimes, the logic may not include all the relevant events, misinterpret key:value relationships, or rely on a flawed understanding of the raw data.
Whatever the case, don’t sweat it - this is completely normal.
Detections aren’t always going to be straightforward, and they’re rarely perfect on the first try. That can generally be attributed to one of three things:
You’re still figuring out the threat behavior, so you widen the scope to better understand what the signal actually looks like.
You’re playing it safe, so you expand the logic to reduce the chance of false negatives - even if it means more noise at first.
You’re waiting on data since the logs haven’t rolled in quite yet, so you’re guessing at the structure of the log.
While each of these cases are different, they share one similarity: uncertainty.
But as a detection engineer, you can’t let perfection delay progress. Start broad, get something in place, and refine as you learn more. That’s how a top-tier detection suite is built.
Exceptions
Exceptions let you preserve your detection logic while suppressing known and expected behavior from surfacing as alerts.
Once an exception is in place, any alert that matches it won’t trigger - saving your team from digging through noise you’ve already deemed as benign.
They’re especially useful for:
Known service account activity
Trusted IP addresses
Confirmed user travel
Regular and verified actions by internal teams
Extra points if you combine multiple fields (e.g. IP + user + process) into a single exception to increase fidelity.
But a word of caution: don’t implement exceptions until you’ve confirmed the activity is always a false positive both with relevant stakeholders and historical data.
It only takes one measly edge case to turn a well-meaning exception into a dangerous blind spot.
Threshold & Rolling Windows
Thresholds and Rolling Windows wor hand-in-hand to fine-tune the “when” behind a detection.
A threshold is the number of times an event needs to happen.
A rolling window defines how quickly it needs to happen to be considered suspicious.
Let’s break that down with an example.
Say you’re building a brute-force detection. You wouldn't want to alert if someone fails a log in 10 times over a week. But 10 failed attempts in 10 minutes? That’s probably worth flagging.
So, you set a threshold of 10 over a rolling window of 10 minutes.
But what if a system already locks accounts after 5 failed attempts in a 10-minute span?
Well, in that case, your detection wouldn’t fire unless you lower your threshold or extend the rolling window (and at this point, you may even go back to the drawing board with the design of this detection as a whole).
In many cases, your threshold might just be 1 if the activity is rare or highly privileged enough to be suspicious on its own. But for other detections, you may need to experiment, monitor performance, and adjust over time as you learn what “normal” looks like.
However, you may not truly know what is needed until a detection has been in production for some time.
Correlations
Sometimes a detection seems logically sound, but then it hits production and you realize it’s not producing the value you expected.
That’s when it might be time to start thinking about correlations.
Correlation detections help you connect multiple lower-fidelity signals into a higher-fidelity alert. You can link different detections within a rolling window to pick up signals that would generally go unnoticed unless being performed in succession.
At first, use cases for correlation detections might not jump out at you. But over time, especially once detections are live, patterns will start to emerge.
And one of the strongest use cases? Insider threat detection.
Let’s say you have a rule watching for large columns of files shared externally. A savvy insider might stay just under the threshold to avoid triggering an alert. But what if that same user also modified admin configurations or deleted artifacts around the same time?
Each activity in isolation may seem benign, but together? It may just be suspicious.
These detections are much more complex than traditional rules, but they’re a powerful tuning strategy when a single log line just isn’t enough to tell the full story.
Alert Context
If a detection fires and no one knows what to do with it, was it even valuable?
As a Detection Engineer, one of your top priorities is to make alerts actionable. Analysts should be able to understand what happened and why it matters within seconds, and be able to investigate further with just a few clicks.
While how alerts are delivered will vary from SIEM to SIEM, the essence remains the same. Here’s how to elevate your alert contexts:
Write Clear, Verbose Titles: The title should be tailored specifically to the alert that just fired and what artifacts are involved;
Bad Title: A user CRUD action occurred in the platform
Good Title: User first.last@website.com deleted an API Key in Sensitive Application via CLI
Include Key Artifacts: Deliver the most important details right in the alert body, ideally as readable JSON or a structured output. Include fields like:
IP Addresses
Network/Location Data
Actor identity
Target resources or users
Behavioral indicators
Provide Tailored Playbooks: Every alert should have relevant, up-to-date playbooks including response steps specific to the detection, escalation paths, and links to relevant documentation or dashboards.
Your ultimate goal is this: analysts should be able to triage and act on a detection with minimal guesswork and uncertainty. If alerts are consistently taking 5+ minutes to triage, they can likely be improved.
Severity Adjustments
Tuning isn’t just about logic or exceptions, it’s also about making sure your severity reflects reality.
Adjusting detection severity up or down is a normal part of the process. It’s also just as common to add dynamic severity logic based on key:value pairs as you start to gain more environmental context.
Maybe a detection didn’t end up being the strong Indicator of Compromise you thought it would be, or maybe you’ve added exceptions or correlations that raise your confidence. Either way, regular severity audits are a must.
And here’s the golden rule: Reserve critical severity for alerts with high confidence and high potential for malicious activity. If everything is critical, then nothing is.
Get in the habit of tuning severity as part of your detection maturation process. It’s not just about catching threats, it’s about helping your team easily prioritize the right ones.
- Today’s Sponsor -
Prepare for a career in Cybersecurity, one sip at a time with The Security Sip. With rapidly evolving threats and technologies, many struggle to gain the right skills and experience to break into the cybersecurity industry. This course is designed to transform beginners into industry-ready professionals over 12 sections, 85 modules, and 155 exercises. Check it out!
An Iterative Detection Lifecycle
I’ve said it before and I’ll say it again: Detection Engineering is an iterative process and your team needs to treat it that way.
You’re not going to get detections perfect the first time, every time. Maybe not even the second or third. But that’s okay.
Just like software development has its lifecycle, so should your detection engineering process. You need to build this into your team’s operating procedures - regularly reviewing what’s working, what’s not, and where improvements are needed.
A scalable detection lifecycle should include three types of reviews:
Regular Reviews: Set a cadence, monthly or quarterly, to review your detection suite as a team. These reviews are your chance to flag underperforming, noisy, or outdated detections that may have gone unnoticed.
Ad-Hoc Reviews: Tuning can’t always wait. You need a fast lane for real-time feedback, whether it’s from your SOC analysts drowning in false positives or engineers spotting a logic mistake. Build a simple process for submitting requests, implementing changes, and peer reviewing.
New Detection Reviews: Every new detection should have a follow-up baked into the process. A week or two after the push to production, circle back and look at the alerts it’s generating, assess its performance, and adjust as needed. Don’t set it and forget it.
Take the time to formalize your detection lifecycle. Align on timelines, responsibilities, and escalation paths. Most importantly, get buy-in across the team.
If you want a detection suite that scales, you need a process that supports it.
Prioritization
Now, I know what you’re thinking: “But Ryan, how do I know which detections actually need tuning?”
Here’s how I approach it:
Look for High-Volume Detections: If a detection is firing at a higher frequency than the rest of your suite, it’s a prime candidate for tuning. High-volume alerts are often the low-hanging fruit, especially early after rolling out your SIEM. In most cases, they’re triggered by common, legitimate user behavior that just hasn’t been refined for your environment yet. Over time, these should become less obvious if you’re tuning effectively.
High False Positive Rate: If you’re constantly marking a detection as false positive, something’s off. Check your logic, maybe it’s too loose. Or maybe it’s time to add an exception for a known, safe activity. High false positive counts waste time and erode the quality of your detection suite.
Low True Positive Rate: This is the next level of detection maturity. Let’s say you’ve got two detections: one has 1 true positive for every 10 false positives, and the other has 1 true positive for every 2 false positives. We’d want to turn our attention to the first detection even if the latter is higher volume overall, and rethink what constitutes a true vs. false positive for this specific use case.
High Time to Remediation: If a specific alert is consistently taking longer to triage, that’s a red flag. It may be a sign the detection isn’t actionable, lacks context, or the playbook is weak. While an underdeveloped SOAR might be part of the issue, slow Mean-time-to-remediation often points to alerts that aren’t clear or concise enough. Revisit your alert contexts and supporting artifacts.
Ad-Hoc Tuning Requests: As mentioned earlier, you need a fast feedback loop from your team triaging alerts on the front lines. While their input might be anecdotal, it can often be the most practical signal for tuning. Build a channel to intake requests, require evidence to be submitted in the form, and watch your detection fidelity slowly form itself to your environment.
Prioritization will always depend on your team’s maturity, but if you’re unsure where to start - this is your playbook.
💬 How do you ensure your team adheres to an iterative detection lifecyle? I’d love to here your leadership and operational tips below!
Don’t Overthink It, Just Do It
At the end of the day, tuning your detection suite comes down to understanding your environment and building processes that scale.
Without frameworks in place to support iteration, it’s easy for the day-to-day grind to overshadow the long-term fidelity of your detections. But if you want a detection program that actually works, one that analysts trust and that surfaces real threats, you need to make tuning part of the culture.
Take pride in your detections. The rules you write aren’t set-and-forget, they’re living pieces of your defensive strategy. And like anything you care about, they deserve routine attention.
You wouldn’t go too long without a car tune-up, so don’t let your detection engine go untouched either.
Securely Yours,
Ryan G. Cox
P.S. The Cybersec Cafe follows a weekly cadence.
Each week, I deliver a Deep Dive on a cybersecurity topic designed to sharpen your perspective, strengthen your technical edge, and support your growth as a professional - straight to your inbox.
. . .
For more insights and updates between issues, you can always find me on Twitter/X or my Website. Let’s keep learning, sharing, and leveling up together.