Templated rule - MFA Rejected by User generating false positives

[MikeP324]

query_data (4).csv
query_data (3).csv
Describe the bug
The templated rule 'MFA Rejected by User' for version 2.0.3 - https://github.com/Azure/Azure-Sentinel/blob/master/Solutions/Microsoft%20Entra%20ID/Analytic%20Rules/MFARejectedbyUser.yaml is generating a lot of false positive noise within our environment.

The code looks for Entra ID 500121 events against the sign-in logs then combines that data with the the UEBA IdentityInfo and BehaviorAnalytics tables. I would like to query the summarize actions for the block of code below.

| join kind=leftouter (
BehaviorAnalytics
| where ActivityType in ("FailedLogOn", "LogOn")
| where isnotempty(SourceIPAddress)
| project UsersInsights, DevicesInsights, ActivityInsights, InvestigationPriority, SourceIPAddress
| project-rename IPAddress = SourceIPAddress
| summarize
UsersInsights = make_set(UsersInsights, 1000),
DevicesInsights = make_set(DevicesInsights, 1000),
IPInvestigationPriority = sum(InvestigationPriority)
by IPAddress)

As a device can have multiple logon events and UEBA records/logs the IP investigation priority score for each record I don't see a need to sum all the records together for the query period. If you use the sum() the reported figure can be 50-60+ or over 100 in some cases. The threshold is 20, all the individual records for the IP investigation priority scores have been between 0-6.

Expected behavior
Should the full code be something like the following?

let riskScoreCutoff = 20; //Adjust this based on volume of results
SigninLogs
| where ResultType == 500121
| extend additionalDetails_ = tostring(Status.additionalDetails)
| extend UserPrincipalName = tolower(UserPrincipalName)
| where additionalDetails_ =~ "MFA denied; user declined the authentication" or additionalDetails_ has "fraud"
| summarize StartTime = min(TimeGenerated), EndTime = max(TimeGenerated), UserId = any(UserId), AADTenantId=any(AADTenantId), DeviceName=any(DeviceDetail.displayName), IsManaged=any(DeviceDetail.isManaged), OS = any(DeviceDetail.operatingSystem) by UserPrincipalName, IPAddress, AppDisplayName
| extend Name = tostring(split(UserPrincipalName,'@',0)[0]), UPNSuffix = tostring(split(UserPrincipalName,'@',1)[0])
| join kind=leftouter (
IdentityInfo
| summarize LatestReportTime = arg_max(TimeGenerated, *) by AccountUPN
| project AccountUPN, Tags, JobTitle, GroupMembership, AssignedRoles, UserType, IsAccountEnabled
| summarize
Tags = make_set(Tags, 1000),
GroupMembership = make_set(GroupMembership, 1000),
AssignedRoles = make_set(AssignedRoles, 1000),
UserType = make_set(UserType, 1000),
UserAccountControl = make_set(UserType, 1000)
by AccountUPN
| extend UserPrincipalName=tolower(AccountUPN)
) on UserPrincipalName
| join kind=leftouter (
BehaviorAnalytics
| where ActivityType in ("FailedLogOn", "LogOn")
| where isnotempty(SourceIPAddress)
| project UsersInsights, DevicesInsights, ActivityInsights, InvestigationPriority, SourceIPAddress
| project-rename IPAddress = SourceIPAddress
| summarize
UsersInsights = make_set(UsersInsights, 1000),
DevicesInsights = make_set(DevicesInsights, 1000)
by IPAddress, IPInvestigationPriority = InvestigationPriority)
on IPAddress
| extend UEBARiskScore = IPInvestigationPriority
| where UEBARiskScore > riskScoreCutoff
| sort by UEBARiskScore desc

I have also changed the first summarize line from

| summarize StartTime = min(TimeGenerated), EndTIme = max(TimeGenerated) by UserPrincipalName, UserId, AADTenantId, IPAddress

to (Enriches the data)

| summarize StartTime = min(TimeGenerated), EndTime = max(TimeGenerated), UserId = any(UserId), AADTenantId=any(AADTenantId), DeviceName=any(DeviceDetail.displayName), IsManaged=any(DeviceDetail.isManaged), OS = any(DeviceDetail.operatingSystem) by UserPrincipalName, IPAddress, AppDisplayName

Screenshots
If applicable, add screenshots to help explain your problem.
Attached two CSV files query_data (3).csv contains the raw data with sensitive information removed and query_data (4).csv contains the output from the triggered incident.

[v-rusraut]

Hi @MikeP324, Thanks for flagging this issue, we will investigate this issue and get back to you with some updates. Thanks!

[MikeP324]

@v-rusraut - Been thinking about this one over the past few days, what are the options for checking for successful MFA event within 5 minutes of the failed message to reduce the noise?

[v-rusraut]

Hi @MikeP324,
We are working with respective team, we will update you.