-
Notifications
You must be signed in to change notification settings - Fork 42
Add "Using EPSS in SSVC" How-To docs #933
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| # EPSS Quantile Binning as an SSVC Amplifier | ||
|
|
||
| !!! tip inline end "" | ||
|
|
||
| !!! tip "Obtain SSVC Data" | ||
|
|
||
| If you are using the [SSVC Deployer Decision Model](../deployer_tree.md), you might already know that | ||
| the [CISA Vulnrichment program](https://github.com/cisagov/vulnrichment) | ||
| provides some SSVC data that is made | ||
| available via the [CVE Services API](https://cveawg.mitre.org/api-docs/). | ||
|
|
||
| !!! tip "Obtain EPSS Data" | ||
|
|
||
| You can get EPSS data from the [EPSS website](https://www.first.org/epss/) | ||
| or use their [API](https://www.first.org/epss/api) to fetch scores programmatically. | ||
|
|
||
| In [another how-to](epss_probability.md), we showed how to use the [Exploit Prediction Scoring System](https://www.first.org/epss/) | ||
| ([EPSS](https://www.first.org/epss)) | ||
| probability scores as one of a few different inputs to inform the | ||
| SSVC [Exploitation](../../reference/decision_points/exploitation.md) decision point. | ||
| This approach can be a useful approach to refine or augment the *input* to an existing SSVC decision model. | ||
|
|
||
| In this how-to, we'll explore a different approach that uses EPSS percentiles | ||
| as an amplifier to adjust the *output* of an existing SSVC decision model. | ||
|
|
||
| ## Starting Out with the SSVC Deployer Decision Model | ||
|
|
||
| Let's start with the assumption that you're already using the | ||
| [SSVC Deployer Decision Model](../deployer_tree.md) and have a basic understanding of SSVC. | ||
| But then after reading | ||
| [Probability, Percentiles, and Binning - How to understand and interpret EPSS Scores](https://www.first.org/epss/articles/prob_percentile_bins), | ||
| you realize that you would like to use EPSS percentiles to amplify the output of your existing decision model. | ||
|
|
||
| !!! question "Why use percentiles instead of raw probabilities?" | ||
|
|
||
| Percentiles provide a relative ranking of vulnerabilities based on their EPSS scores. | ||
| This can be particularly useful when you want to prioritize vulnerabilities in the context of the entire vulnerability landscape. | ||
| By using percentiles, you can identify which vulnerabilities are more likely to be exploited _compared to others_. | ||
| The [EPSS SIG blog](https://www.first.org/epss/articles/prob_percentile_bins) discusses some of the tradeoffs involved in using percentiles versus raw probabilities. | ||
|
|
||
| That said, we are using percentiles for this how-to simply because we already | ||
| we showed how to use probabilities in [another how-to](epss_probability.md). | ||
| Either approach can be valid depending on your specific needs and context. | ||
|
|
||
| One straightforward way to use EPSS is to create bins based on the EPSS score | ||
| and use these bins as amplifiers in your SSVC decision-making process. | ||
|
|
||
| !!! tip "See Also" | ||
|
|
||
| See the sidebar in [this how-to](epss_probability.md) for a discussion of the tradeoffs involved in binning. | ||
|
|
||
| ## Binning Percentiles | ||
|
|
||
| SSVC provides several basic decision points that bin percentiles into discrete quantiles. | ||
| Expand the example below to see the currently available options. | ||
|
|
||
| ??? example "Exploring Decision Points for Binning Percentiles" | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.doc_helpers import example_block | ||
| from ssvc.decision_points.basic.quantiles import DECISION_POINTS as _DPS | ||
|
|
||
| for dp in _DPS.values(): | ||
| print(example_block(dp)) | ||
| ``` | ||
|
|
||
| Now, your primary concern is to ensure that you are addressing the | ||
| vulnerabilities that are most likely to be exploited. | ||
| In conversations with your organization's risk owners, you determine that | ||
| they'd like to apply a policy that is consistent with the following: | ||
|
|
||
| - If the EPSS percentile is significantly higher than the median, the vulnerability | ||
| should be prioritized two levels higher than the default SSVC recommendation. | ||
| - If the EPSS percentile is above the median but not significantly so, the vulnerability | ||
| should be prioritized one level higher than the default SSVC recommendation. | ||
| - If the EPSS percentile is significantly lower than the median, the vulnerability | ||
| should be deprioritized lower than the default SSVC recommendation. | ||
|
|
||
| You can achieve this with the `quartiles` decision point: | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.decision_points.basic.quantiles.quartiles import LATEST as QUARTILES | ||
| from ssvc.doc_helpers import example_block | ||
|
|
||
| print(example_block(QUARTILES)) | ||
| ``` | ||
|
|
||
| !!! warning inline end "We're not saying this is a good rule set!" | ||
|
|
||
| The rules given here are just an example to illustrate how you might use EPSS percentiles | ||
| as an amplifier in your decision-making process. | ||
| We are not suggesting that this is a good idea or that you should follow these rules. | ||
| In fact, these rules are rather aggressive and could result in fully _half_ of all | ||
| vulnerabilities being prioritized _higher_ than they would without the EPSS data. | ||
| Your organization's risk owners should determine the appropriate policy for your context, | ||
| but we would recommend something considerably less aggressive than this example. | ||
|
|
||
| You decide to apply the following rules: | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.decision_tables.example.epss_quartile import update_map_doc | ||
|
|
||
| print(update_map_doc()) | ||
| ``` | ||
|
|
||
| ## Building the Decision Table | ||
|
|
||
| Given the rules outlined above, you build a new Decision Table that takes the | ||
| default outcome from the | ||
| [SSVC Deployer Decision Model](../deployer_tree.md) and applies the EPSS quartile | ||
| information to amplify it. The resulting decision table looks like this: | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.decision_tables.example.epss_quartile import EXAMPLE as DT | ||
| from ssvc.decision_tables.helpers import dt2df_md | ||
|
|
||
| print(dt2df_md(DT)) | ||
| ``` | ||
|
|
||
| A diagram of the decision model is shown below. | ||
|
|
||
| ???+ example "Example Decision Table Diagram" | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.decision_tables.example.epss_quartile import EXAMPLE as DT | ||
| from ssvc.decision_tables.helpers import mapping2mermaid, mermaid_title_from_dt | ||
|
|
||
| rows = DT.mapping | ||
| title = mermaid_title_from_dt(DT) | ||
| print(mapping2mermaid(rows, title=title)) | ||
| ``` | ||
|
|
||
| And here is a JSON object representation of the decision table for programmatic use: | ||
|
|
||
| ??? example "Example Decision Table JSON" | ||
|
|
||
| The JSON representation of the decision table is shown below. | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.decision_tables.example.epss_quartile import EXAMPLE as DT | ||
|
|
||
| print("```json") | ||
| print(DT.model_dump_json(indent=2)) | ||
| print("```") | ||
| ``` | ||
|
|
||
| Now you can use this decision table in your SSVC implementation to adjust | ||
| the prioritization of vulnerabilities based on their EPSS percentiles. | ||
|
|
||
| ## Conclusion | ||
|
|
||
| In this how-to, we've demonstrated how to use EPSS percentiles as an amplifier | ||
| to adjust the output of an existing SSVC decision model. | ||
| While the example provided is considerably more aggressive than we would recommend | ||
| in practice, it illustrates one way to incorporate EPSS data into an established SSVC-based | ||
| vulnerability management strategy. | ||
| By incorporating statistical insights from EPSS, you can prioritize | ||
| vulnerabilities more effectively based on their likelihood of exploitation. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,173 @@ | ||
| # Combining EPSS with other Exploitation-Related Decision Points | ||
|
|
||
| SSVC users might want to combine exploitation-related information from multiple | ||
| sources into a single decision point for use downstream in a decision table | ||
| such as the SSVC [Deployer Decision Model](../deployer_tree.md). | ||
|
|
||
| One such source is the [Exploit Prediction Scoring System](https://www.first.org/epss/) | ||
| ([EPSS](https://www.first.org/epss)) probability score. | ||
|
|
||
| !!! question "What is the EPSS Probability Score?" | ||
|
|
||
| The EPSS probability score is a number between 0 and 1 that indicates the likelihood of | ||
| a vulnerability being exploited in the wild within the next 30 days. | ||
|
|
||
| ## Other Exploitation-Related Information Sources | ||
|
|
||
| However, EPSS is not the only source of exploitation-related information. | ||
| The | ||
| [CISA Known Exploited Vulnerabilities (KEV) catalog](https://www.cisa.gov/known-exploited-vulnerabilities-catalog) | ||
| is another important source. | ||
| Additional exploitation-related information can be found in the | ||
| [CVSS Exploit Maturity](https://www.first.org/cvss/specification-document#Exploit-Maturity-E) | ||
| vector element. | ||
|
|
||
| We have implemented SSVC Decision Points to reflect both CISA KEV and CVSS Exploit Maturity: | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.doc_helpers import example_block | ||
| from ssvc.decision_points.cvss.exploit_maturity import LATEST as CVSS_E | ||
| from ssvc.decision_points.cisa.in_kev import LATEST as CISA_KEV | ||
|
|
||
| for dp in [CISA_KEV, CVSS_E]: | ||
| print(example_block(dp)) | ||
| ``` | ||
|
|
||
| !!! note inline end "EPSS on Probability Binning" | ||
|
|
||
| In a [blog post](https://www.first.org/epss/articles/prob_percentile_bins) on the | ||
| [EPSS website](https://www.first.org/epss), the EPSS SIG discusses the challenges of | ||
| binning probabilities. | ||
|
|
||
| !!! quote "EPSS SIG on Binning" | ||
|
|
||
| However, there are a number of problems with binning. Bins are, by construction, subjective transformations of, in this case, a cardinal probability scale. And because the bins are subjectively defined, there is room for disagreement and misalignment across different users. There is no universal "right" answer to what the cut off should be between a high, and medium, or medium and low. | ||
|
|
||
| Moreover, arbitrary cutoffs force two scores, which may be separated by the tiniest of a value, to be labeled and then handled differently, despite there being no practical difference between them. For example, if two bins are set and the cutoff is set at 0.5, two vulnerabilities with probabilities of 0.499 and 0.501 would be treated just the same as two vulnerabilities with probabilities of 0.001 and 0.999. This kind of range compression is unavoidable and so any benefits from this kind of mental shortcut must be weighed against the information loss inevitable with binning. | ||
|
|
||
| For these reasons, EPSS does not currently bin EPSS scores using labels. | ||
|
|
||
| From a data _provider_ perspective, this makes sense. | ||
| Avoiding information loss early in the information pipeline is a good idea. | ||
| However, from a data _consumer_ perspective, and especially when one is making | ||
| a choice between a finite number of options (as in SSVC), binning can be a useful | ||
| tool to reduce the complexity of the decision space. | ||
|
|
||
| ## Binning Probabilities | ||
|
|
||
| We have also provided a few basic SSVC Decision Points to capture probability-based | ||
| information in different ways. | ||
| Because SSVC is based on categorical decision points, we need to bin the | ||
| continuous probability scores into discrete categories. | ||
| However, as the EPSS SIG points out (see sidebar), there are *always* tradeoffs | ||
| involved in binning. | ||
| That's why we provide several different options for binning probabilities so that | ||
| SSVC users can choose one that best fits their needs (or create their own if | ||
| none of the provided options is suitable). | ||
| Expand the example below to see the currently available options. | ||
|
|
||
| ??? example "Exploring Decision Points for Binning Probabilities" | ||
|
|
||
| We provide a few different decision points based on probability bins. | ||
| You might look these over and choose one that fits your needs. | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.doc_helpers import example_block | ||
| from ssvc.decision_points.basic.probability import DECISION_POINTS as _DPS | ||
|
|
||
| for dp in _DPS.values(): | ||
| print(example_block(dp)) | ||
| ``` | ||
|
|
||
| For this example, let's say you decide to use the *Probability Scale in 5 weighted levels, ascending* | ||
| decision point: | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.doc_helpers import example_block | ||
| from ssvc.decision_points.basic.probability.five_weighted import LATEST as DP | ||
|
|
||
| print(example_block(DP)) | ||
| ``` | ||
|
|
||
| With our exploitation and probability binning decision points in hand, | ||
| we can now consider how to combine them in a decision table to get | ||
| a more nuanced view of exploitation risk. | ||
|
|
||
| ## Designing an Exploitation-focused Decision Table | ||
|
|
||
| Let's say you decide to create a new Decision Table that combines the | ||
| EPSS probability information with the other exploitation-related decision | ||
| points to determine a more informed outcome using the SSVC [Exploitation](../../reference/decision_points/exploitation.md) decision point. | ||
|
|
||
| As a reminder, the SSVC [Exploitation](../../reference/decision_points/exploitation.md) decision point has the following values: | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.doc_helpers import example_block | ||
| from ssvc.decision_points.ssvc.exploitation import LATEST as EXP | ||
|
|
||
| for dp in [EXP]: | ||
| print(example_block(dp)) | ||
| ``` | ||
|
|
||
| In conversations with your organization's risk owners, you determine that you | ||
| should focus your vulnerability management efforts on the vulnerabilities | ||
| that are either already being actively exploited or are very likely to be exploited soon. | ||
|
|
||
| You decide to apply the following rules: | ||
|
|
||
| | Rule | Description | | ||
| |------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||
| | Prioritize vuls in KEV as Active | If the vulnerability is in the CISA KEV, set the SSVC [Exploitation](../../reference/decision_points/exploitation.md) value to *Active*. | | ||
| | Treat very high EPSS probabilities as already Active | If the EPSS probability is >90%, set SSVC [Exploitation](../../reference/decision_points/exploitation.md) value to *Active*. | | ||
| | Amplify high EPSS probabilities | If the EPSS probability is 75-90%, bump the SSVC [Exploitation](../../reference/decision_points/exploitation.md) value up one category across the board. | | ||
| | Assume Public PoCs are Active when EPSS probability is more likely than not. | If the EPSS probability is 55-75%, bump SSVC [Exploitation](../../reference/decision_points/exploitation.md) = *Public PoC* to *Active* | | ||
| | Default: Use CVSS Exploit Maturity | By default, use the [CVSS Exploit Maturity](../../reference/decision_points/cvss/exploit_maturity.md) value to set the SSVC [Exploitation](../../reference/decision_points/exploitation.md) value, unless one of the other rules apply. | | ||
|
|
||
| After constructing the decision table according to these rules, you end up with the following table of values: | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.decision_tables.example.epss_percentile import EXAMPLE as DT | ||
| from ssvc.decision_tables.helpers import dt2df_md | ||
|
|
||
| print(dt2df_md(DT)) | ||
| ``` | ||
|
|
||
| A diagram of the decision model is shown below. | ||
|
|
||
| ???+ example "Example Decision Table Diagram" | ||
|
|
||
| The diagram below shows the decision model for this example. | ||
| Each path through the diagram corresponds to a row in the table above. | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.decision_tables.example.epss_percentile import EXAMPLE as DT | ||
| from ssvc.decision_tables.helpers import mapping2mermaid, mermaid_title_from_dt | ||
|
|
||
| rows = DT.mapping | ||
| title = mermaid_title_from_dt(DT) | ||
| print(mapping2mermaid(rows, title=title)) | ||
| ``` | ||
|
|
||
| And here is a JSON object representation of the decision table for programmatic use: | ||
|
|
||
| ??? example "Example Decision Table JSON" | ||
|
|
||
| The JSON representation of the decision table is shown below. | ||
|
|
||
| ```python exec="true" idprefix="" | ||
| from ssvc.decision_tables.example.epss_percentile import EXAMPLE as DT | ||
| print("```json") | ||
| print(DT.model_dump_json(indent=2)) | ||
| print("```") | ||
| ``` | ||
|
|
||
| Now you've created a clear way to combine EPSS probability scores with other | ||
| exploitation-related information to inform your SSVC decisions downstream. | ||
|
|
||
| ## Conclusion | ||
|
|
||
| In this How-To, we've explored how to combine EPSS probability scores with other | ||
| exploitation-related information in an SSVC decision table. | ||
| By thoughtfully designing decision points and tables, you can create a more nuanced | ||
| and effective vulnerability management strategy that prioritizes risks based on | ||
| the likelihood of exploitation. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| # EPSS → SSVC Intro | ||
|
|
||
| !!! quote inline end "The [FIRST EPSS SIG](https://www.first.org/epss) Explains [EPSS](https://www.first.org/epss/model)" | ||
|
|
||
| EPSS is a daily estimate of the probability of exploitation activity being observed over the next 30 days | ||
|
|
||
| The [Exploit Prediction Scoring System](https://www.first.org/epss) ([EPSS](https://www.first.org/epss)) is a statistical model that estimates the likelihood of a vulnerability being exploited in the wild. EPSS can be a valuable input when assessing the exploitation risk associated with vulnerabilities. EPSS provides two key metrics: | ||
|
|
||
| - **EPSS Score**: A score between 0 and 1 indicating the probability of exploitation. | ||
| - **EPSS Percentile**: A ranking percentile that indicates how the EPSS score compares to other vulnerabilities. | ||
|
|
||
| In the following pages, we'll demonstrate a few different ways to incorporate EPSS data into your SSVC decision models. | ||
|
|
||
| <div class="grid cards" markdown> | ||
|
|
||
| - :material-dice-multiple: [Using EPSS Probability Scores as an Input to SSVC Decisions](epss_probability.md) | ||
|
|
||
| --- | ||
|
|
||
| How to use EPSS probability scores to augment other data sources to | ||
| inform the SSVC Exploitation decision point. | ||
|
|
||
| - :material-percent: [Using EPSS Percentiles to Amplify SSVC Decisions](epss_percentiles.md) | ||
|
|
||
| --- | ||
|
|
||
| How to use EPSS percentiles to amplify the output of an existing SSVC decision model. | ||
|
|
||
| </div> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.