Custom Detection Rules in Microsoft Defender XDR: Step-by-Step Guide with 5 Ready-to-Use Rules

You've written a great advanced hunting query. It finds encoded PowerShell, or phishing links that users clicked. But a query only helps when someone runs it. Custom detection rules in Microsoft Defender fix that: they run your KQL query on a schedule, create alerts when it finds something, and can even respond automatically, for example by isolating a device or soft-deleting an email.

This guide shows how custom detection rules work, how to create one step by step, the columns your query must return, and five ready-to-use rules for real security scenarios.

Before you start: you need permission to manage security settings in the Defender portal (for example, Security Administrator) and the licenses for the tables your query uses. New to KQL? Read our KQL for beginners guide first.

What is a custom detection rule?

A custom detection rule is a saved advanced hunting query that Microsoft Defender XDR runs automatically. When the query returns results, Defender creates alerts, groups them into incidents with Microsoft's own detections, and optionally takes response actions on the devices, users, emails or files involved.

Think of it as turning your best hunting queries into your own detections, tuned to your environment.

Requirements for the query

A rule can only be created when the query returns the columns Defender needs to build an alert:

• Timestamp: used as the alert time.
• An event identifier: for Defender for Endpoint tables, include DeviceId and ReportId from the same event.
• At least one entity column, such as DeviceId, DeviceName, AccountObjectId, AccountUpn, RecipientEmailAddress or SenderFromAddress. These become the "impacted assets" you can take action on.
• For email actions, include NetworkMessageId and RecipientEmailAddress.

Tip: if you use summarize, the required columns disappear unless you keep them. Use arg_max(Timestamp, *) or arg_max(Timestamp, ReportId, DeviceId) to keep the latest event's values. Microsoft's documentation lists the full set of supported columns.

How to create a custom detection rule (step by step)

1. Open the Microsoft Defender portal and go to Investigation & response > Hunting > Advanced hunting.
2. Write your query and run it. Check the results make sense and aren't too noisy.
3. Click Create detection rule (above the query editor).
4. Alert details: enter a clear name (for example, "Encoded PowerShell execution"), choose the frequency, severity and category, add MITRE ATT&CK techniques, and write a description and recommended actions for the analyst who receives the alert.
5. Impacted entities: choose which columns identify the main device, user or mailbox.
6. Actions (optional): select automatic responses, such as isolating the device or soft-deleting emails.
7. Scope: apply the rule to all devices or specific device groups.
8. Review and create. The rule appears under Hunting > Custom detection rules, where you can edit, disable or run it.

How often should the rule run?

You can choose a frequency such as every 24 hours, 12 hours, 3 hours or every hour, and for supported tables a near-real-time (continuous) option. Each frequency looks back over a matching window of data, so you don't miss events between runs.

• Every hour or continuous: urgent threats such as ransomware behavior or RMM tools on servers.
• Every 3 to 12 hours: phishing clicks, suspicious inbox rules.
• Every 24 hours: hygiene and slow-moving checks, such as accounts with risky settings.

Automated actions you can take

Entity	Example actions
Devices	Isolate device, collect investigation package, run antivirus scan, initiate automated investigation, restrict app execution
Files	Quarantine file, allow or block
Users	Mark user as compromised, disable user, force password reset (depending on your setup)
Emails	Move to a folder or delete (soft delete)

Be careful: start with alerts only. Add automatic actions after the rule has run for a week or two without false positives. Isolating a production server because of a noisy rule is a bad day for everyone.

5 custom detection rules for real-world threats

1. Encoded PowerShell execution

DeviceProcessEvents
| where Timestamp > ago(1h)
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where ProcessCommandLine has_any ("-enc ", "-EncodedCommand", "FromBase64String")
| project Timestamp, DeviceId, DeviceName, ReportId, AccountName, AccountUpn = InitiatingProcessAccountUpn, ProcessCommandLine

Attackers hide malicious scripts in Base64. Some admin tools do this too, so exclude known-good scripts by path or hash after testing. Severity: Medium. Frequency: every hour.

2. Remote access tools (RMM) installed

DeviceProcessEvents
| where Timestamp > ago(1h)
| where FileName has_any ("ScreenConnect", "AnyDesk", "TeamViewer", "Atera", "Splashtop")
| where InitiatingProcessFileName !in~ ("your-approved-deployment-tool.exe")
| project Timestamp, DeviceId, DeviceName, ReportId, AccountName, FileName, ProcessCommandLine, FolderPath

Phishing campaigns increasingly install legitimate remote-access tools instead of malware. Remove the tools your company actually uses from the list. Severity: High.

3. Users who clicked through to a phishing link

UrlClickEvents
| where Timestamp > ago(3h)
| where IsClickedThrough == true
| where ThreatTypes has "Phish"
| project Timestamp, ReportId, AccountUpn, Url, NetworkMessageId, IPAddress

Safe Links warned the user, and they clicked through anyway. These users need a password check and sign-in review right away. Severity: High.

4. Inbox rule that forwards or redirects mail

CloudAppEvents
| where Timestamp > ago(3h)
| where ActionType in ("New-InboxRule", "Set-InboxRule")
| where RawEventData has_any ("ForwardTo", "RedirectTo", "ForwardAsAttachmentTo")
| project Timestamp, ReportId, AccountObjectId, AccountDisplayName, IPAddress, ActionType

Attackers who take over a mailbox often add a rule to forward mail to themselves or hide replies. Severity: High.

5. One internal sender emailing many colleagues

EmailEvents
| where Timestamp > ago(1h)
| where EmailDirection == "Intra-org"
| summarize Recipients = dcount(RecipientEmailAddress),
            arg_max(Timestamp, ReportId, NetworkMessageId, Subject)
  by SenderFromAddress
| where Recipients > 50
| project Timestamp, ReportId, SenderFromAddress, NetworkMessageId, Subject, Recipients

A hijacked mailbox sending phishing to the whole company is a common next step. Adjust the threshold to your organization's normal behavior, and exclude HR or communications mailboxes that send bulk mail. Learn more about the columns in our EmailEvents table guide.

Best practices for custom detections

• Test first: run the query over the last 7 to 30 days and count how many alerts it would create.
• Name and describe rules clearly: the analyst at 2 AM should know what the alert means and what to do.
• Tune exclusions in the query, not by ignoring alerts. Document why each exclusion exists.
• Map to MITRE ATT&CK so you can see which techniques you cover.
• Review rules every quarter: check alert counts, false positives and whether tables or columns have been renamed.
• Keep a library: store your rule queries in Git or a shared document so they survive staff changes.

Troubleshooting

Problem	Fix
"Create detection rule" is greyed out or shows missing columns	Add Timestamp, ReportId and an entity column (and DeviceId for device tables) to the results
Rule never fires	Run the query manually with a longer time range; check the rule is enabled and the scope includes the right devices
Too many alerts	Raise thresholds, add exclusions, or reduce the frequency
Actions not available	The required entity column for that action (for example, DeviceId or NetworkMessageId) is missing

Where to get rule ideas

Start from investigations you've already done. Our 20 Microsoft Defender XDR advanced hunting queries cover phishing, identity and endpoint scenarios that make excellent detection rules once you add the required columns. Our real case of device code phishing on Microsoft 365 is another good source of detection ideas.

FAQ

What are custom detection rules in Microsoft Defender XDR?

They are advanced hunting queries that run on a schedule and create alerts, and optionally take automatic response actions, when they return results.

What columns are required for a custom detection rule?

Timestamp, an event identifier such as ReportId (with DeviceId for device tables), and at least one entity column like DeviceId, AccountObjectId or RecipientEmailAddress.

Can custom detection rules take automatic actions?

Yes. Depending on the entities in the results, rules can isolate devices, quarantine files, act on users or move and delete emails.

Are custom detection rules the same as Microsoft Sentinel analytics rules?

They're similar ideas in different products. Custom detections run in the Defender portal on Defender data; Sentinel analytics rules run on your Sentinel workspace and can include data from many other sources.