Ranking Detection Rules with the Wilson Score Interval

Detection engineering teams often inherit a deceptively simple problem: how do you decide which detection rules are actually your best? Most teams already collect two useful metrics for every rule. First is precision—the percentage of alerts that analysts ultimately determine are legitimate security activity. Second is alert volume over a given period.

Unfortunately, these metrics often point in different directions.

A rule with 100% precision across 10 alerts sounds fantastic, but should you trust that result? Maybe. Maybe not. Ten alerts simply isn’t much evidence.

A rule with 100% precision across 1,000 alerts is a very different story. At that point, you’ve accumulated enough observations to be genuinely confident in its quality.

At the other extreme, a rule producing 50,000 alerts with only 25% precision is consuming an enormous amount of analyst time while generating mostly noise.

The challenge is that precision and volume are difficult to compare independently. Detection engineering needs a single score that rewards both high precision and sufficient evidence. Fortunately, statistics solved this problem years ago. It’s called the Wilson Score Interval, and it’s surprisingly practical for ranking detection rules.

The Wilson Score Interval

The Wilson Score Interval was originally developed for evaluating binary outcomes such as pass/fail experiments. More recently, it has found its way into production systems for ranking user-generated content. Reddit famously used it for comment ranking because it prevents comments with only a handful of upvotes from immediately floating to the top. Yelp and other review platforms have used similar approaches for ranking businesses.

The same idea maps naturally to detection engineering.

Each alert has two possible outcomes: it was a true positive or it wasn’t. Precision is simply the proportion of alerts that were ultimately confirmed as real. The question becomes:

Given the evidence we have so far, what’s the lowest precision this rule is likely to have?

Instead of ranking rules by their observed precision, the Wilson Score ranks them by the lower bound of the confidence interval. In other words, it asks, “What’s the worst this rule probably is, given the amount of evidence we’ve collected?”

That makes the metric intentionally conservative. A rule doesn’t earn a high ranking simply because it got lucky with a small sample size.

The formula looks intimidating at first glance, but each component has a straightforward meaning.

score = (p̂ + z²/2n − z√(p̂(1−p̂)/n + z²/4n²)) / (1 + z²/n)

Where:

• p̂ is the observed precision expressed as a decimal (85% becomes 0.85).
• n is the total number of alerts used to calculate that precision.
• z is 1.96, the standard z-score corresponding to a 95% confidence interval.

It’s worth calling out that z isn’t something you calculate from your data. It’s a tuning constant you choose based on how conservative you want the ranking to be. Using 1.96 is the conventional choice because it represents 95% confidence and provides a good balance between caution and practicality.

A Few Examples

The behavior becomes much easier to understand with real numbers.

The resulting ranking is:

Rule A > Rule D > Rule B > Rule C

At first, Rule B surprises people. It has perfect precision, so why doesn’t it beat Rule D?

Because ten alerts simply don’t provide enough evidence to be confident that the rule will continue performing perfectly. The Wilson Score intentionally discounts small sample sizes. Rule D has slightly lower precision, but it has demonstrated that performance across 500 alerts. That’s enough evidence to place much greater confidence in the result.

Rule A is exactly what we’d hope to see. It combines exceptional precision with substantial volume, producing an almost perfect score.

Rule C demonstrates the opposite extreme. Fifty thousand alerts certainly provide plenty of evidence—but unfortunately, the evidence shows that the rule is noisy. The Wilson Score doesn’t reward volume by itself. It rewards confidence in quality.

This balance between observed performance and statistical confidence is exactly what makes the metric useful in operational environments.

Operational Applications

One immediate use is identifying retirement candidates.

Suppose you decide that any rule with a Wilson Score below 0.3 after generating more than 500 alerts deserves review. Rather than relying on anecdotal complaints from analysts, you now have an objective threshold for identifying rules that consistently generate poor outcomes. Whether you tune them or retire them becomes an engineering discussion backed by data instead of intuition.

The opposite is true for newly deployed rules.

A brand-new detection with only a handful of alerts will naturally receive a relatively low Wilson Score, even if every alert is a true positive. That’s not a flaw—it’s the entire point. The score isn’t saying the rule is bad. It’s saying you don’t yet have enough evidence to be confident. Those are the rules to keep watching, not immediately discard.

Wilson Scores can also improve analyst workflows.

If your SOC routes alerts differently based on rule quality, rules with consistently high Wilson Scores deserve faster escalation paths because history suggests they’re trustworthy. Conversely, alerts generated by low-scoring rules deserve a bit more skepticism before consuming additional investigation time.

The metric is also useful when prioritizing tuning work.

Many teams already sort rules by precision to identify noisy detections. Adding Wilson Score makes that process far more defensible. Filtering to rules with more than 100 alerts and then examining those with the lowest Wilson Scores naturally surfaces the rules where tuning will likely have the greatest operational impact.

Strategic Applications

Wilson Scores become even more valuable when viewed across an entire detection program.

Instead of looking at individual rules, plot the distribution of scores across your rule library every quarter. Mature detection programs should gradually shift toward higher-confidence detections. Seeing more rules clustered between 0.7 and 0.99, and fewer between 0.1 and 0.3, provides a meaningful measure of program health.

The metric also helps justify investments.

If rules built on a particular telemetry source consistently produce higher Wilson Scores than rules built elsewhere, that suggests the data source is generating high-quality detections. Expanding that telemetry becomes a data-driven investment decision rather than a subjective preference. Likewise, a data source that continually produces low-confidence detections may not justify its ingestion and storage costs.

Wilson Scores also pair nicely with MITRE ATT&CK reporting.

Many organizations proudly report how much of the ATT&CK matrix they cover, but coverage alone says little about detection quality. A stronger story is showing that your highest-confidence rules—perhaps those scoring above 0.7—provide meaningful coverage across critical ATT&CK tactics. Coverage quality matters just as much as coverage quantity.

Finally, Wilson Scores improve executive reporting.

Reporting that your organization maintains 847 active detection rules doesn’t tell leadership much about effectiveness.

Reporting that 62% of your rules score above 0.7 on Wilson confidence, up from 41% last year, demonstrates measurable improvement in detection quality. That’s the kind of metric that reflects engineering maturity rather than inventory size.

Know What the Metric Doesn’t Tell You

Like any metric, Wilson Score has limitations.

First, alert volume is inherently time-dependent. A rule that’s been running for three years has naturally accumulated more evidence than one deployed last month. That additional volume tends to increase confidence. If you’re comparing rules of significantly different ages, consider normalizing your analysis using consistent observation windows or alerts per day.

Second, Wilson Score only measures confidence in precision. It says nothing about recall. A rule can score 0.99 while still missing half of the attacks occurring in your environment. The metric tells you how trustworthy the alerts you receive are—not what threats you might be failing to detect.

Finally, the score is only as reliable as your analyst disposition data. If analysts apply verdicts inconsistently, or if a significant portion of alerts never receive a final disposition, then precision becomes noisy. Wilson Score faithfully reflects that uncertainty because it’s built directly on those underlying observations.

A Better Way to Rank Detection Rules

If your SOC already tracks alert volume and analyst dispositions, you have everything you need to start using the Wilson Score Interval today.

The implementation is straightforward enough for a spreadsheet and only takes a few lines of Python if you’d rather calculate it programmatically.

The real value isn’t the formula itself. It’s having a single, statistically grounded metric that combines precision and evidence into one comparable score. Once every rule lives on the same scale, it becomes much easier to prioritize tuning work, identify retirement candidates, evaluate new detections, measure program maturity, and communicate meaningful progress to leadership.

Sometimes the most impactful improvements in detection engineering don’t come from building new detections. They come from making better decisions about the ones you already have.