WIP: update new analysis

Author: hp23 Server

_hp/hp/tools/analysis/analysis_december_2024.ipynb

@@ -59,6 +59,9 @@
     "# Parsing + Basic 5x+ each, and reruns (such that all tests have 5+ results)\n",
     "# Ubuntu 22.04, Android 11, MacOS 14.3.1, iPadOS 17.3.1\n",
     "# All runs performed between 2024-03-07 and 2024-05-14\n",
+    "\n",
+    "# New browsers: Ubuntu 22.04, MacOS 15.2\n",
+    "# All runs performed between 2024-12-13 and 2024-12-20\n",
     "initial_data = \"\"\"\n",
     "SELECT \"Result\".*, \n",
     "\"Response\".raw_header, \"Response\".status_code, \"Response\".label, \"Response\".resp_type,\n",
@@ -118,7 +121,8 @@
    },
    "source": [
     "# Overview\n",
-    "- Collected between 875800 and 1072632 results for 12 browsers"
+    "- Collected between 875800 and 1072632 results for 12 browsers (original)\n",
+    "- Collected between X and Y results for 4 browsers (update december)"
    ]
   },
   {
@@ -136,8 +140,9 @@
     "# opera (same as chrome, only mobile: 63\n",
     "# chrome (HSTS testing only): 70\n",
     "# brave (android shields disabled): 65\n",
+    "# chrome (test): 71\n",
     "\n",
-    "df = df.loc[~df[\"browser_id\"].isin([30, 27, 26, 54, 28, 66, 63, 44, 70, 65])]"
+    "df = df.loc[~df[\"browser_id\"].isin([30, 27, 26, 54, 28, 66, 63, 44, 70, 65, 71])]"
    ]
   },
   {
@@ -2598,7 +2603,9 @@
   {
    "cell_type": "markdown",
    "id": "12f42fc8-2f18-48a7-be7a-ba913fa915e5",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true
+   },
    "source": [
     "### General\n",
     "- Basic:\n",
@@ -3584,7 +3591,9 @@
   {
    "cell_type": "markdown",
    "id": "12e83670-77c8-4c13-addf-c03253a62e59",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true
+   },
    "source": [
     "### Test groups that are the most unstable\n",
     "- Overall mean: 0.07% of tests have more than one outcome\n",
@@ -4861,7 +4870,9 @@
   {
    "cell_type": "markdown",
    "id": "45c8c774-4bbc-4d8b-beb6-03a820dc8cd8",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true
+   },
    "source": [
     "### Majority Voting\n",
     "- Perform majority voting\n",
@@ -19855,7 +19866,9 @@
   {
    "cell_type": "markdown",
    "id": "795a4f14-e376-40f7-88a2-2d48d88e17d9",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true
+   },
    "source": [
     "### Firefox 121 vs 122 vs 123 (desktop linux)\n",
     "- subresourceloadingCOEP_img_direct both basic and parsing\n",
@@ -20127,7 +20140,7 @@
     "jp-MarkdownHeadingCollapsed": true
    },
    "source": [
-    "### Safari (MacOS 17.3.1) vs Chrome/WebKit (iPadOS 17.3.1)\n",
+    "### Safari (MacOS 14.3.1) vs Chrome/WebKit (iPadOS 17.3.1)\n",
     "- **Differences: perfAPI (mixed content blocking differences), accesswindow_direct (204 + different download behavior), subresourceloadingCORP (mixed-content/auto-upgrade?)**\n",
     "- Noise: subresourceloadingCOEP"
    ]
@@ -20303,6 +20316,100 @@
     "display(r)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "4cbc754f-1e27-497f-8b86-b2dcd9f1a8d6",
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true
+   },
+   "source": [
+    "## Browser Comparisons New Update!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b2bbd3e0-73bf-4fa5-969a-73efbeb213df",
+   "metadata": {},
+   "source": [
+    "### Safari (MacOS 14.3.1) vs Safari (MacOS 15.2)\n",
+    "- ??"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fd153860-4158-41a6-9719-ffc94afa8ad5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "r = com_browsers(bf, \"safari macOS 15.2 18.2 selenium real\", \"safari macOS 14.3.1 17.3.1 selenium real\")\n",
+    "display(r[\"test_id\"].apply(lambda x: x.split(\"_\")[0]).value_counts())\n",
+    "display(r)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "24413de8-42be-407f-a34e-03e202d6422f",
+   "metadata": {},
+   "source": [
+    "### Chrome (122) vs Chrome (131)\n",
+    "- ??"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c0b70f62-0f9a-4b33-b86a-ea87f575b878",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "r = com_browsers(bf, \"chrome Ubuntu 22.04 122 selenium headless-new\", \"chrome Ubuntu 22.04 131 selenium headless-new\")\n",
+    "display(r[\"test_id\"].apply(lambda x: x.split(\"_\")[0]).value_counts())\n",
+    "display(r)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9ea818ab-eb12-4557-8845-1d49da695a67",
+   "metadata": {},
+   "source": [
+    "### Firefox (123) vs Firefox (133)\n",
+    "- ??"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6bcde80d-12b4-4b36-ab1f-ffe78e455f29",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "r = com_browsers(bf, \"firefox Ubuntu 22.04 123 selenium headless-new\", \"firefox Ubuntu 22.04 133 selenium headless-new\")\n",
+    "display(r[\"test_id\"].apply(lambda x: x.split(\"_\")[0]).value_counts())\n",
+    "display(r)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "32c7859d-3a5d-472e-bbd5-7d53715ad0f4",
+   "metadata": {},
+   "source": [
+    "### Brave (v1.62.156 (Chromium 121)) vs Brave (v1.73.101 (Chromium 131))\n",
+    "- ??"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8361b430-dfb7-4900-9880-f7c1d78db3a1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "r = com_browsers(bf, \"brave Ubuntu 22.04 v1.62.156 (121.0.6167.139) selenium headless-new\", \"brave Ubuntu 22.04 v1.73.101 (131.0.6778.139) selenium headless-new\")\n",
+    "display(r[\"test_id\"].apply(lambda x: x.split(\"_\")[0]).value_counts())\n",
+    "display(r)"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "1c8d5dd9-0e56-445f-8907-90b51a63ff88",