-
Notifications
You must be signed in to change notification settings - Fork 10.4k
[CF1] app confidence scorecards #25755
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
a121090
[CF1] app confidence scorecards
deadlypants1973 972a6bf
Update src/content/docs/cloudflare-one/applications/app-library.mdx
deadlypants1973 c42f442
Update src/content/docs/cloudflare-one/applications/app-library.mdx
deadlypants1973 b7066f5
Update src/content/docs/cloudflare-one/applications/app-library.mdx
deadlypants1973 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -47,3 +47,61 @@ The Shadow IT Discovery dashboard will provide more details for discovered appli | |
| The App Library synchronizes application review statuses with approval statuses from the [Shadow IT Discovery SaaS analytics](/cloudflare-one/insights/analytics/shadow-it-discovery/) dashboard. | ||
|
|
||
| <Render file="app-library-review-apps" product="cloudflare-one" /> | ||
|
|
||
| ## Application confidence scorecards | ||
|
|
||
| Application confidence scorecards provide automated risk assessment for AI and SaaS applications to help organizations make informed decisions about application approval and security policies. These scores bring scale and automation to the labor- and time-intensive task of evaluating generative AI and SaaS applications. | ||
|
|
||
| The scoring system evaluates applications across multiple security, compliance, and operational dimensions to generate two complementary scores: the Application Posture Score and the Generative AI Posture Score. These scores help security teams identify risks in Shadow AI and Shadow IT deployments without manual auditing of every application. | ||
|
|
||
| To view an application's confidence scorecard: | ||
|
|
||
| 1. In [Zero Trust](https://one.dash.cloudflare.com/), go to **My team** > **App Library** | ||
| 2. Find the application you would like to review or search it by name. | ||
| 3. Review the Application Posture Score and the Generative AI Posture Score which are generated on the application card. | ||
|
|
||
| ### Scoring methodology | ||
| #### Application Posture Score (5 points) | ||
|
|
||
| The Application Posture Score evaluates SaaS providers across five major categories. | ||
|
|
||
| | Category | Points | Assessment Criteria | Scoring Logic | | ||
| |-------------------------------------|:-------:|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||
| | Security and Privacy Compliance | 1.2 | Presence of SOC 2 and ISO 27001 certifications, which signal operational maturity and adherence to security frameworks. | Full credit awarded for both certifications; partial credit for one certification; no credit if neither certification is present. | | ||
| | Data Management Practices | 1.0 | Data retention windows and whether the provider shares data with third parties. | Shorter retention periods and no third-party data sharing earn the highest marks. Applications with indefinite data retention or extensive data sharing receive lower scores. | | ||
| | Security Controls | 1.0 | Support for Multi-Factor Authentication (MFA), Single Sign-On (SSO), TLS 1.3, role-based access controls, and session monitoring capabilities. | These represent table stakes of modern SaaS security. Full credit requires comprehensive support across all controls; partial credit awarded for subset implementation. | | ||
| | Security Reports and Incident History | 1.0 | Availability of trust or security pages, active bug bounty programs, incident response transparency, and recent breach history. | Recent material breaches result in full point deduction. Proactive security measures like bug bounty programs and transparent incident reporting increase scores. | | ||
| | Financial Stability | 0.8 | Company financial status, funding levels, and operational stability. | Public companies and heavily capitalized providers score highest, while startups with limited funding or companies in financial distress receive lower scores. | | ||
| | Total Points | 5.0 | | | | ||
|
|
||
| #### Generative AI Posture Score (5 points) | ||
|
|
||
| | Category | Points | Assessment Criteria | Scoring Logic | | ||
| |---------------------------|:-------:|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||
| | Compliance | 1.0 | Presence of ISO 42001 certification for AI management systems. | Full credit for ISO 42001 certification; no credit without this specialized AI governance certification. | | ||
| | Deployment Security Model | 1.0 | Whether application access requires authentication and implements rate limiting, or if services are publicly exposed without controls. | Authenticated access with proper rate limiting receives full credit; publicly exposed services without controls receive minimal scoring. | | ||
| | System Card | 1.0 | Publication of model or system cards documenting safety evaluations, bias testing, and risk assessments. | Comprehensive system cards with detailed safety and bias documentation receive full credit; incomplete or missing documentation results in score reduction. | | ||
| | Training Data Governance | 2.0 | Whether user data is explicitly excluded from model training and availability of opt-in/opt-out controls for training data usage. | Explicit exclusion of user data from training receives maximum points; opt-in/opt-out controls receive partial credit; no controls or guaranteed user data training receives minimal scoring. | | ||
| | **Total Points** | **5.0** | | | | ||
|
|
||
| ### Automated scoring infrastructure | ||
|
|
||
| #### Web crawling and data extraction | ||
|
|
||
| The scoring system employs automated infrastructure to crawl and analyze public information sources. | ||
|
|
||
| - Data sources: Trust centers, privacy policies, security pages, compliance documents, and vendor documentation. | ||
| - Extraction process: Large language models parse documents to identify relevant information, with structured extraction methods to resist hallucinations and ensure accuracy. | ||
| - Validation requirements: Source validation and structured data extraction prevent false positives and ensure reliable scoring. | ||
|
|
||
| #### Human oversight and quality assurance | ||
|
|
||
| Automated results are supplemented with manual review to maintain transparency and ensure data integrity. | ||
|
|
||
| - Review process: Every automated score undergoes review and audit by Cloudflare analysts before publication in the Application Library. | ||
| - Validation methodology: Combination of automated crawling/extraction with human validation ensures comprehensive and trustworthy scoring. | ||
| - Update frequency: Scores update dynamically as vendors improve security and compliance postures, providing live assessment rather than static reports. | ||
|
|
||
| #### Report score inaccuracies | ||
|
|
||
| If you believe one of the Application confidence scores is incorrect or have additional evidence that should be considered in the scoring process, contact `[email protected]`. Include relevant documentation or evidence that supports your assessment to help us review and update the score accordingly. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.