Machine learning user risk score calculations
Updated August 11, 2021
Advanced adaptive capability powered by machine learning prevents bad actors from impersonating authorized users to gain access to a protected site. Attackers can compromise credentials in any of the following ways:
Social engineering or phishing tactics
Obtain credentials beforehand from a source on the dark web
Compromise a different third-party organization
Create new credentials within an organization they've already breached
Use machine learning to track and analyze the login behavior patterns of authorized users for a period of time to identify their normal login patterns. Each user is assigned a risk score based on their current login behavior in comparison to previous similar login behavior patterns.
A user's risk score might fluctuate based on login event factors. For example, when user Jose attempts to log in, his login behavior pattern is compared to the pattern of the authorized user that Jose claims to be. A mismatch between patterns assigns a higher risk score to Jose. In turn, Jose might be prompted to enter additional authentication requirements or denied access to the protected site.
Prerequisites
SecureAuth Identity Platform release 19.07 or later
SecureAuth Prevent package license for the user risk score feature; contact SecureAuth Support
Machine learning analysis
Machine learning analyzes information gathered from a user's login behavior pattern during a period of time and assigns a risk score to that user to dictate how to handle an authentication request. This risk score is based on whether the user is a low, medium, or high risk user, when compared to other users with similar login behavior patterns. A user's risk score falls within a range of 0 to 100, with 100 indicating a high risk score.
A risk score can change if a user has a different login behavior pattern in the past compared to the present. For example, if during the past two months Trinh consistently logged in at about 9:00 a.m., and now logs in at 2:00 a.m., her risk score increases and Trinh must enter a second factor to authenticate.
Note
SailPoint IdentityIQ and Exabeam UEBA users: Risk scores from these third-party platforms are calculated apart from the SecureAuth user risk score. (More on this later.)
User score components
A user risk score is assigned using the five components listed below. Although the components measure different behaviors, they all calculate deviations from the norm; the farther away the calculation moves from the norm, the higher the user risk score. If the user risk score moves high enough, the user must log in with a second factor or might be denied access.
The following information describes the 5 components that contribute to the user risk score.
Usual time of day the user logs in
All users have a safe time range automatically calculated in accordance with their typical successful login times. (This accounts for small deviations, like getting stuck in traffic or stopping to pick up a cup of coffee.) If a user logs in at an hour that's out of the range, the risk is calculated according to how far the time is from the safe range.
Usual days of the week the user logs in
The day the user logs in is measured against the days the user normally logs in.
Usual level of activity during the last hour
The activity level of the user during the last hour is measured against the user's daily average activity during the last six months. If activity during the last hour is above the six month average, a risk score is calculated proportionate to the difference.
For example, Sima has a daily average activity of 10 successful logins over the last six months, then one day, 20 login attempts occur in the last hour. The SecureAuth User Risk Scoring Service compares the difference and gives Sima a score of 100 to match the 100% more login attempts in the last hour when compared to the past six months.
Variation in the percentage of successful logins
The percentage of successful transactions for the week is measured against the same percentage for the previous six months. If the variation for the week decreases from the six month average, the difference is calculated as a higher user risk.
Failed authentication attempts
The percentage of the user's authentication failure based on the data for the current week.
User risk score calculations
The SecureAuth User Risk Scoring Service calculates a user risk score in real-time by accessing user behavior described in the section above. The final score is a value between 0 and 100, where 0 is the least risk and 100 is the most risk. After scoring, components 1-5 are then weighted, as shown in the following image.
 |
The following information shows how components 1 - 5 are calculated to achieve the user risk score.
Components 1, 2, and 3 represent 85% of the final score.
The components are not individually weighted. If any of the three have a score, the final score is the sum (not the average) of the three.
Example 1: If components 1 and 2 each receive a score of 5 and component 3 receives a score of 10, the final score is 20. The final score, 20, is then multiplied by .85 for a final value of 17. If 17 is a number in the low risk threshold set by the admin, then the user skips entering a second factor.
Example 2: If components 1, 2, and 3 each receive a score of 100, the final score is 300, but is limited to 100 (risk maximum). The final score, 100, is then multiplied by .85 for a final value of 85. If 85 is a number in the medium risk threshold set by the admin, then the user might be redirected to a realm or URL to enter a second factor.
Components 4 and 5 represent 15% of the final score.
The scoring is the same as in the previous examples, except the final score is multiplied by .15.
The final score from each is added to obtain the user risk score.
Component weights stated above, 85% and 15%, cannot be changed.
Data retrieval insights
The SecureAuth Identity Platform gathers end user authentication general statistics for all SecureAuth packages. The SecureAuth Prevent package license for the user risk score feature is an added and adaptive layer of protection.
The SecureAuth User Risk Scoring Service retrieves data immediately after you set user risk settings in the Identity Platform New Experience Policies, Risk Providers page, or an Identity Platform Classic Experience realm.
Whenever you add new end users to the Identity Platform, they will be assessed at a higher risk for approximately 2 - 3 weeks before the risk scores normalize. During this time, new end users will need to enter their second factors more often until their login behavior is established. Remember that all new end users must establish login behavior for 2 - 3 weeks whenever they are added, even to an established site.
The following image shows weekly data retrieval for new end users and when the data is used meaningfully to adjust risk scores for the norm. After you've set the user risk scores and risk thresholds, all data retrieval and risk score adjustments occur automatically.
 |
After three weeks of data retrieval you should see risk scores normalize. If the risk scores for your organization are still high, contact SecureAuth Support to check the machine learning logs. See Set the SecureAuth user risk score actions for important information about blocking rules.
Build user risk settings for your organization
Now that you understand the calculations that drive user risk scores, you're ready to set up the risk tolerance and actions for your organization. This section is meant to help you pull all the risk score pieces together so you can obtain the results your organization expects.
Scenario
In this scenario, the administrator is setting up user risk score thresholds and the actions that will occur within those thresholds.
The scenario shows how to:
Set risk range thresholds for high, medium, and low risk ranges, and how they are impacted by the user score components discussed above.
Set user actions to occur for the risk thresholds you've set up.
Add a third-party provider and understand the impact that has on risk ranges.
Add the SecureAuth Pre-Login Assessment Survey, available to Login for Windows and Login for Mac customers, and understand the impact that has on risk ranges.
Set up the SecureAuth User Risk score threshold ranges. The default threshold numbers are:
Low = 0 - 49; medium = 50 - 89; high = 90 - 100
In the Identity Platform New Experience, on the left side, click Risk Providers.
The user risk threshold settings are global and used by the specified risk provider.
If you have threshold ranges set in Classic Experience realms, they will inherit the settings you make in the New Experience. Wherever you set the threshold ranges last, those ranges will be used in both experiences.
You can set Medium and High thresholds only. To set a different Minimum setting, you must use the Classic Experience.
You can disable the Classic Experience user risk settings for a realm that doesn't require user risk settings. (Realms serve different purposes; you can disable user risk settings on realms that were created for a purpose other than user risk.)
In an Identity Platform Classic Experience realm, on the Adaptive Authentication tab, edit the User Risk section.
The user risk threshold settings are global and used by the specified risk provider.
If you have threshold ranges set in the New Experience, they will inherit the settings you make in the Classic Experience. Wherever you set the threshold ranges last, those ranges will be used in both experiences.
You can set Low, Medium, and High thresholds, where the Low threshold is the same as the Minimum threshold in the New Experience.
Set the SecureAuth user risk score actions.
User risk actions are conditions that occur because they fall within the threshold range of an end user score, for example, Block, Prompt for MFA, Skip MFA, etc.
The user risk score actions are like doors that are opened when the correct number unlocks them. The number is calculated by the sum of components (1-3 or 4 and 5) multiplied by the percent (85% or 15%) compared against the numbers set for the low, medium, and high thresholds. The score will fall within one of the low, medium, or high ranges, and a risk score action "door" opens, determining which action Adaptive Authentication will take.
In the Identity Platform New Experience, on the left side, click Policies, add a new rule, and select User Risk.
When set, if a user risk score is within the threshold range (risk level), users will be prompted for MFA or can skip MFA or will be blocked, depending on the settings.
Warning
SecureAuth strongly recommends not blocking users based on their machine learning scores. Currently, you cannot unlock a user account blocked because of the machine learning user risk score. Blocked users will see the following message:
If you intend to set a blocking rule, be sure its threshold is on the low end of high. Then, if end users are blocked, you can raise the threshold to unblock end users.
The Identity Platform Classic Experience offers 7 user risk score actions. After you set user risk score actions for a policy in the New Experience, any user risk actions you have set in the Classic Experience will be ignored.
If your organization needs one of the user risk score actions available in the Classic Experience, such as Redirect to realm or URL, you can add it to a hybrid deployment. You must set this up in a Classic Experience realm and edit the web.config file to include the action you need. (Contact SecureAuth Support for assistance.)
Caution
After you edit the web.config file, if you change the policy in the New Experience and save your changes, this will overwrite the manual changes on the web.config file. You must add the changes again.
The following user risk score actions are available in the Identity Platform Classic Experience:
Continue Adaptive Authentication
Refuse authentication request
Same as "Block" in the New Experience
Skip to post-authentication
Redirect to realm or URL
Resume authentication workflow
Skip two-factor authentication
Same as "Skip MFA" in the New Experience
Require two-factor authentication
Same as "Prompt for MFA" in the New Experience
Optional: Add a third-party provider, such as SailPoint IdentityIQ or Exabeam UEBA.
Risk scores from third-party providers are calculated apart from the SecureAuth user risk score. You will set up the thresholds for third-party tools the same way you would for the SecureAuth User Risk. The default threshold numbers for third-party providers are:
Low = 0 - 100; medium = 101 - 200; high = 201 - 300
You can use the Secureauth User Risk provider and one or more third-party risk provider thresholds, but the higher risk always triggers the risk score action.
Example: In this scenario, both SecureAuth and SailPoint providers are enabled. The SecureAuth User Risk maximum high risk is set to 100. The SailPoint maximum high risk is set to 300. The risk score actions are set the same for both providers, as shown in the following images.
Example of Identity Platform New Experience Policies and Risk Providers settings
Example of Identity Platform Advanced Settings Adaptive Authentication tab, User Risk settings
End user Kara gets a score of 90 from SecureAuth User Risk and a score of 150 from SailPoint, and is refused authentication even though 90 is lower than 150. Because 90 falls in the SecureAuth high risk category and 150 falls in the SailPoint medium risk category, she is refused authentication; the higher risk (not score) triggers the risk action.
Optional: Integrate the SecureAuth Pre-Login Assessment Survey.
If you are using Login for Windows release 20.09.01+ or Login for Mac release 20.09.02+, you can set up a pre-login questionnaire and then integrate it with the Identity Platform. End users will then see the questionnaire and provide answers before logging in.
You will add or clone a realm in the Identity Platform Classic Experience to add this risk assessment layer. There is no default threshold numbers for the pre-login questionnaire; you must set them according to your organization's risk tolerance.
You must disable other risk providers.
Example: In this scenario, a company called Acme has set a custom pre-login questionnaire and enabled the Acme User Risk provider. All other providers are disabled. The risk thresholds and risk score actions are shown in the follow image:
End user Ashok is working remotely from home. He receives a score of 25 and is redirected to an Acme COVID questionnaire. He answers the questions and the questionnaire logic determines that he is a low risk based on his answers. Ashok is able to authenticate in and continue with his work.