cloudflare · G4brym · Feb 10, 2025 · Jan 31, 2025
@@ -0,0 +1,40 @@
+---
+title: Expanded AI insights in Cloudflare Radar
+description: Expanded AI insights with new API endpoints for Internet services rankings, robots.txt, and inference data
+products:
+  - radar
+date: 2025-02-04T11:00:00Z
+---
+
+[**Radar**](/radar/) has expanded its AI insights with new API endpoints for Internet services rankings, robots.txt analysis, and AI inference data.
+
+#### Internet services ranking
+
+Radar now provides [rankings for Internet services](/radar/glossary/#internet-services-ranking), including Generative AI platforms, based on anonymized 1.1.1.1 resolver data.
+Previously limited to the annual Year in Review, these insights are now available daily via the [API](/api/resources/radar/subresources/ranking/subresources/internet_services/), through the following endpoints:
+
+- [`top`](/api/resources/radar/subresources/ranking/subresources/internet_services/methods/top/) show service popularity at a specific date.
+- [`timeseries_groups`](/api/resources/radar/subresources/ranking/subresources/internet_services/methods/timeseries_groups/) track ranking trends over time.
+
+#### Robots.txt
+
+Radar now analyzes [robots.txt](/radar/glossary/#robotstxt) files from the top 10,000 domains, identifying AI bot access rules.
+AI-focused user agents from [ai.robots.txt](https://github.com/ai-robots-txt/ai.robots.txt) are categorized as:
+
+- **Fully allowed/disallowed** if directives apply to all paths (`*`).
+- **Partially allowed/disallowed** if restrictions apply to specific paths.
+
+These insights are now available weekly via the [API](/api/resources/radar/subresources/robots_txt/), through the following endpoints:
+
+- [`top/user_agents/directive`](/api/resources/radar/subresources/robots_txt/subresources/top/subresources/user_agents/methods/directive/) to get the top AI user agents by directive.
+- [`top/domain_categories`](/api/resources/radar/subresources/robots_txt/subresources/top/methods/domain_categories/) to get the top domain categories by robots.txt files.
+
+#### Workers AI
+
+Radar now provides insights into public AI inference models from [Workers AI](/workers-ai/), tracking usage trends across **models** and **tasks**.
+These insights are now available via the [API](/api/resources/radar/subresources/ai/subresources/inference/), through the following endpoints:
+
+- [`summary`](/api/resources/radar/subresources/ai/subresources/inference/subresources/summary/) to view aggregated `model` and `task` popularity.
+- [`timeseries_groups`](/api/resources/radar/subresources/ai/subresources/inference/subresources/timeseries_groups/) to track changes over time for `model` or `task`.
+
+Learn more about the new Radar AI insights in our [blog post](https://blog.cloudflare.com/expanded-ai-insights-on-cloudflare-radar/).
@@ -5,6 +5,12 @@ productLink: "/radar/"
 productArea: Developer platform
 productAreaLink: /workers/platform/changelog/platform/
 entries:
+  - publish_date: "2025-02-04"
+    title: Add Internet services ranking, robots.txt, and AI inference endpoints
+    description: |-
+      * Added [Internet services ranking endpoints](/api/resources/radar/subresources/ranking/subresources/internet_services/).
+      * Added [robots.txt endpoints](/api/resources/radar/subresources/robots_txt/).
+      * Added [AI inference endpoints](/api/resources/radar/subresources/ai/subresources/inference/).
   - publish_date: "2024-06-27"
     title: Change TCP connection tampering API endpoints to TCP Resets Timeouts
     description: |-
@@ -25,11 +31,11 @@ entries:
       * Added Layer 7 Summaries by `http_method`, `http_version`, `ip_version`, `managed_rules`, `mitigation_product`.
       * Added Layer 7 Timeseries Groups by `http_method`, `http_version`, `ip_version`, `managed_rules`, `mitigation_product`, `industry`, `vertical`.
       * Added Layer 7 Top by `industry`, `vertical`.
-      * Deprecated Layer 7 [timeseries groups without dimension](/api/resources/radar/subresources/attacks/subresources/layer7/subresources/timeseries_groups/methods/get/).
+      * Deprecated Layer 7 timeseries groups without dimension.
         * To continue getting this data, switch to the new
           [timeseries group by mitigation_product](/api/resources/radar/subresources/attacks/subresources/layer7/subresources/timeseries_groups/methods/mitigation_product/)
           endpoint.
-      * Deprecated Layer 7 [summary without dimension)](/api/resources/radar/subresources/attacks/subresources/layer7/subresources/summary/methods/get/).
+      * Deprecated Layer 7 summary without dimension.
         * To continue getting this data, switch to the new
           [summary by mitigation_product](/api/resources/radar/subresources/attacks/subresources/layer7/subresources/summary/methods/mitigation_product/)
           endpoint.
@@ -55,15 +61,11 @@ entries:
         [timeseries](/api/resources/radar/subresources/attacks/subresources/layer3/methods/timeseries/) endpoint
         to start using the new Layer 3 data source by default, fetching the old data source now requires sending the parameter
         `metric=bytes_old`.
-      * Deprecated Layer 3
-        [summary](/api/resources/radar/subresources/attacks/subresources/layer3/subresources/summary/methods/get/) endpoint, this will stop
-        receiving data after 2023-08-14.
+      * Deprecated Layer 3 summary endpoint, this will stop receiving data after 2023-08-14.
         * To continue getting this data, switch to the
           new [timeseries group protocol](/api/resources/radar/subresources/attacks/subresources/layer3/subresources/summary/methods/protocol/)
           endpoint.
-      * Deprecated Layer 3
-        [timeseries groups](/api/resources/radar/subresources/attacks/subresources/layer3/subresources/timeseries_groups/methods/get/)
-        endpoint, this will stop receiving data after 2023-08-14.
+      * Deprecated Layer 3 timeseries groups endpoint, this will stop receiving data after 2023-08-14.
         * To continue getting this data, switch to the
           new [timeseries group protocol](/api/resources/radar/subresources/attacks/subresources/layer3/subresources/timeseries_groups/methods/protocol/)
           endpoint.
@@ -76,11 +78,11 @@ entries:
   - publish_date: "2023-07-20"
     title: Add URL Scanner endpoints
     description: |-
-      * Added [urlscanner endpoints](/api/resources/url_scanner/subresources/scans/methods/list/). For more information, refer to [URL Scanner](/radar/investigate/url-scanner/).
+      * Added [urlscanner endpoints](/api/resources/url_scanner/). For more information, refer to [URL Scanner](/radar/investigate/url-scanner/).
   - publish_date: "2023-06-20"
     title: Add Quality endpoints
     description: |-
-      * Added [quality endpoints](/api/resources/radar/subresources/quality/subresources/iqi/methods/summary/).
+      * Added [quality endpoints](/api/resources/radar/subresources/quality/).
   - publish_date: "2023-06-07"
     title: Add BGP stats, pfx2as and moas endpoint
     description: |-
@@ -95,8 +97,8 @@ entries:
   - publish_date: "2023-03-20"
     title: Add AS112 and email endpoints
     description: |-
-      * Added [AS112 endpoints](/api/resources/radar/subresources/as112/subresources/summary/methods/dnssec/).
-      * Added [email endpoints](/api/resources/radar/subresources/email/subresources/security/subresources/summary/methods/arc/).
+      * Added [AS112 endpoints](/api/resources/radar/subresources/as112/).
+      * Added [email endpoints](/api/resources/radar/subresources/email/).
   - publish_date: "2023-01-23"
     title: Updated IPv6 calculation method
     description: |-

@@ -1,7 +1,7 @@
 ---
 pcx_content_type: navigation
 title: API reference
-external_link: /api/resources/radar/subresources/annotations/subresources/outages/methods/get/
+external_link: /api/resources/radar/
 sidebar:
   order: 6
 

@@ -66,7 +66,7 @@ of the table are defined as follows:
 * `Confidence`: The level of confidence that we have on the event being a true hijacks. Values can be `High`, `Medium`, or `Low`.
 * `Tags`: The relevant evidence presented as short tags, presenting key facts we compiled using additional data sources, such as RPKI validation results or network relationship.
 
-You can also access the detection result programmatically via our [public API](https://developers.cloudflare.com/api/resources/radar/subresources/bgp/subresources/hijacks/subresources/events/methods/list/) ([CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) license).
+You can also access the detection result programmatically via our [public API](/api/resources/radar/subresources/bgp/subresources/hijacks/subresources/events/methods/list/) ([CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) license).
 
 ## Certificates
 
@@ -90,13 +90,13 @@ Cloudflare Speed Test measures latency multiple times over the course of the tes
 
 ## Content categories
 
-Cloudflare uses a variety of data sources to categorize domains. Using Cloudflare Radar, you can view the content categories associated with a given domain. Cloudflare customers using Cloudflare Gateway or 1.1.1.1 for Families can decide to block certain categories, like "Adult Content", in addition to security threats like malware and phishing.
+Cloudflare uses a variety of data sources to categorize domains. Using Cloudflare Radar, you can view the content categories associated with a given domain. Cloudflare customers using Cloudflare Gateway or [1.1.1.1 for Families](https://one.one.one.one/family/) can decide to block certain categories, like "Adult Content", in addition to security threats like malware and phishing.
 
 In some cases, a domain may be miscategorized. For example, a social media site might be categorized as "Shopping & Auctions". If you believe a domain is miscategorized, or a domain has not yet been categorized, please provide your suggested category using [this form](https://radar.cloudflare.com/domains/feedback) to bring it to our attention.
 
 ## Domain rankings
 
-Domain Rankings is based on our anonymized and aggregated 1.1.1.1 resolver data, complies with our [privacy policy](https://www.cloudflare.com/en-gb/privacypolicy/), and aims to identify the top most popular domains that reflect how people use the Internet globally. Domain Rankings’ popularity metric is best described as the estimated number of unique users that access a domain over some period of time.
+Domain Rankings is based on our anonymized and aggregated [1.1.1.1 resolver](https://one.one.one.one/dns/) data, complies with our [privacy policy](https://www.cloudflare.com/en-gb/privacypolicy/), and aims to identify the top most popular domains that reflect how people use the Internet globally. Domain Rankings’ popularity metric is best described as the estimated number of unique users that access a domain over some period of time.
 
 Trending domains are domains which are currently experiencing an increase in popularity. Domains Trending Today are domains spiking in popularity, reflecting increased interest potentially related to a particular event or a topic. Domains Trending This Week are domains that have steadily grown in popularity, reflecting an increase of their user base over the week.
 
@@ -112,7 +112,7 @@ An early warning signal that an Internet outage may be underway on a given netwo
 
 ## Internet services ranking
 
-Internet services ranking is based on our anonymized and aggregated 1.1.1.1 resolver data, complies with our [privacy policy](https://www.cloudflare.com/en-gb/privacypolicy/), and aims to identify the top most popular Internet services that reflect how people use the Internet globally. A service represents one or more domains aggregated together. Ranking popularity metric is best described as the estimated number of unique users that access domains associated with a service, over some period of time.
+Internet services ranking is based on our anonymized and aggregated [1.1.1.1 resolver](https://one.one.one.one/dns/) data, complies with our [privacy policy](https://www.cloudflare.com/en-gb/privacypolicy/), and aims to identify the top most popular Internet services that reflect how people use the Internet globally. A service represents one or more domains aggregated together. Ranking popularity metric is best described as the estimated number of unique users that access domains associated with a service, over some period of time.
 
 ## Internet traffic trends
 
@@ -162,7 +162,7 @@ The Post-Quantum Encryption Adoption graph shows the share of HTTPS requests to
 
 A [robots.txt](https://www.cloudflare.com/learning/bots/what-is-robots-txt/) file contains instructions for bots that tell them which webpages they can and cannot access.
 
-The data displayed for robots.txt is based on successfully parsed robots.txt files from the top 10,000 domains. From these files, we count the occurrences of each user agent under the "Allow" and "Disallow" directives. A user agent is classified as "fully allowed" or "fully disallowed" if the directive value is "*". Otherwise, if the user agent is only allowed or disallowed to crawl specific paths, it is classified as "partially allowed" or "partially disallowed."
+The data displayed for robots.txt is based on successfully parsed robots.txt files from the top 10,000 domains. From these files, we count the occurrences of each user agent under the `Allow` and `Disallow` directives. A user agent is classified as "fully allowed" or "fully disallowed" if the directive value is `*`. Otherwise, if the user agent is only allowed or disallowed to crawl specific paths, it is classified as "partially allowed" or "partially disallowed."
 
 Currently, we only include AI-focused user agents listed in the [ai.robots.txt](https://github.com/ai-robots-txt/ai.robots.txt) repository.