Skip to content

Commit 0b9bf98

Browse files
author
hp23 Server
committed
Add MacOS analysis
1 parent 3fa3295 commit 0b9bf98

File tree

1 file changed

+172
-10
lines changed

1 file changed

+172
-10
lines changed

‎_hp/hp/tools/analysis/analysis_december_2024.ipynb‎

Lines changed: 172 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -20449,18 +20449,40 @@
2044920449
"## Browser Comparisons New Update!"
2045020450
]
2045120451
},
20452+
{
20453+
"cell_type": "code",
20454+
"execution_count": 188,
20455+
"id": "72dcdc72-db0a-4f7b-ba70-5fa6874e586f",
20456+
"metadata": {},
20457+
"outputs": [],
20458+
"source": [
20459+
"# TODO: show all tests that now have unique behaviors in the new browser versions! i.e., they did not change the behavior to one of another browser but to something new!\n",
20460+
"# ..."
20461+
]
20462+
},
2045220463
{
2045320464
"cell_type": "markdown",
2045420465
"id": "b2bbd3e0-73bf-4fa5-969a-73efbeb213df",
20455-
"metadata": {},
20466+
"metadata": {
20467+
"jp-MarkdownHeadingCollapsed": true
20468+
},
2045620469
"source": [
2045720470
"### Safari (MacOS 14.3.1) vs Safari (MacOS 15.2)\n",
20458-
"- Lot's of differences, mostly to do with HTTP(S)?\n",
20459-
"- Inital glance: HSTS maybe bug (regression) with casing of header; in general: network library has changed (\\x00, \\r, \\n, space, : etc in headers now leads to error when it did not before and the other way round!)\n",
20460-
"- Start:\n",
20461-
"- End:\n",
20462-
"- Time taken:\n",
20463-
"- TODO: analyze them and test them!"
20471+
"- Start: 13:47\n",
20472+
"- End: 16:06\n",
20473+
"- Time taken: 1h30m\n",
20474+
"- Total of 1866 diffs\n",
20475+
"- **TODO** ~3 new bug reports to WebKit!\n",
20476+
"- Changes:\n",
20477+
" - **TODO report** New behavior: `:<header>: <value>` is a network error in WebKit, was skipped before (and it other browsers)\n",
20478+
" - New WebKit behavior (related to row 3 and 1): `\\r<header> | <header>\\n` and similar are now network error in WebKit (before they were skipped and/or accepted which is still the case in other browsers)\n",
20479+
" - **TODO report** New behavior (bug?!): `<whitespace><header>: <value>` is now allowed in WebKit\n",
20480+
" - New WebKit behavior (row 4): `<header>\\t: <value>` rows are now ignored (same as Firefox)\n",
20481+
" - New WebKit behavior (row 8/9): mixed image autoupgrades (TAO and subresourceloading/CSP is affected)\n",
20482+
" - New WebKit behavior (row 28): TAO and 302 fixed\n",
20483+
" - New WebKit behavior (row 20): NULL and fetch fixed\n",
20484+
" - **TODO report** New WebKit behavior (bug?!): HSTS only works with devtools open??\n",
20485+
" - Some noise/strange results for subresourceloadingCORP_img: could be a caching (or timeout) issue in both new or old Safari or both"
2046420486
]
2046520487
},
2046620488
{
@@ -20650,14 +20672,87 @@
2065020672
},
2065120673
{
2065220674
"cell_type": "code",
20653-
"execution_count": 167,
20675+
"execution_count": 231,
20676+
"id": "af54da7d-b6aa-4474-b956-699205bf243b",
20677+
"metadata": {},
20678+
"outputs": [],
20679+
"source": [
20680+
"# New general parsing change(s)\n",
20681+
"# Related to 1-5 in Table 5\n",
20682+
"# (no entry yet?, new behavior only in WebKit) :<header>: <value> is now \"network error\" in WebKit \"failed to load resource cannot parse response\", such rows are simply ignored/skipped in Firefox/Chrome (old WebKit)\n",
20683+
"# (similar to row 3 and 1, but network error) \\r<header> | <header>\\r | <head \\r er> | <header>\\n all now result in \"network error\"\n",
20684+
"# fullscreen_iframe_direct 30/30\n",
20685+
"# fullscreen_iframe_child_allow 15/15\n",
20686+
"# sniffing_script_direct 8/16 \n",
20687+
"# acccesswindow_direct_direct 4/20\n",
20688+
"# referrer_iframe_iframe 15/23\n",
20689+
"# fetch_GET_credentials 19/41 (similar numbers for all other fetch_* tests)\n",
20690+
"# script_execution_iframe_direct 26/42\n",
20691+
"# framing_iframe_nested 26/42 (similar numbers for other framing tests)\n",
20692+
"# subresourceloadingCORP_img_direct 34/51\n",
20693+
"# imgloading_iframe_direct 30/67\n",
20694+
"# subresourceloadingCOEP_img_direct 30/67\n",
20695+
"# perfAPI_img_direct 2+16=18/82\n",
20696+
"# framing_iframe_direct 52/84\n",
20697+
"# upgradeHSTS_subdomain_subdomain 11/291\n",
20698+
"# upgradeHSTS_direct_direct 31/828\n",
20699+
"\n",
20700+
"# (no entry yet, new behavior only in WebKit) <whitespace><header>: <value> is now allowed in WebKit (such rows are skipped in other browsers)\n",
20701+
"# sniffing_script_direct 2/16\n",
20702+
"# accesswindow_direct_direct 4/20\n",
20703+
"# referrer_iframe_iframe 2/23\n",
20704+
"# fetch_GET_credentials 3/41 (similar numbers for all other fetch_* tests)\n",
20705+
"# script_execution_iframe_direct 4/42\n",
20706+
"# framing_iframe_nested 4/42 (similar numbers for other framing tests)\n",
20707+
"# subresourceloadingCORP_img_direct 4/51\n",
20708+
"# imgloading_iframe_direct 4/67\n",
20709+
"# perfAPI_img_direct 2/82\n",
20710+
"# framing_iframe_direct 8/84\n",
20711+
"\n",
20712+
"# (row 4?) <header> : | <header>\\t: -> now results in ignoring the header (same as Firefox)\n",
20713+
"# sniffing_script_direct 6/16\n",
20714+
"# accesswindow_direct_direct 12/20\n",
20715+
"# referrer_iframe_iframe 6/23\n",
20716+
"# fetch_GET_credentials 19/41 (similar numbers for all other fetch_* tests)\n",
20717+
"# script_execution_iframe_direct 12/42\n",
20718+
"# framing_iframe_nested 12/42 (similar numbers for other framing tests)\n",
20719+
"# subresourceloadingCORP_img_direct 13/51\n",
20720+
"# imgloading_iframe_direct 6/67\n",
20721+
"# perfAPI_img_direct 6/82\n",
20722+
"# framing_iframe_direct 24/84\n",
20723+
"\n",
20724+
"# (row 8 or 9?) automated upgrade of mixed-content images\n",
20725+
"# imgloading_iframe_direct 27/67\n",
20726+
"# subresourceloadingCOEP_img_direct 9/67\n",
20727+
"# perfAPI_img_direct 16+28=44/82\n",
20728+
"\n",
20729+
"# row 28: TAO and 302 fixed\n",
20730+
"# perfAPI_img_direct 12/82\n",
20731+
"\n",
20732+
"# row 20: NULL in Header Values (Fetch)\n",
20733+
"# upgradeHSTS_subdomain_subdomain: 8/291\n",
20734+
"# upgradeHSTS_direct_direct: 8/828\n",
20735+
"\n",
20736+
"# (no entry yet) HSTS caching issue? with devtools closed\n",
20737+
"# Example upgradeHSTS_subdomain_subdomain_http_sub.headers.websec.saarland_https_sub.headers.websec.saarland_134_basic\n",
20738+
"# upgradeHSTS_subdomain_subdomain ~270/291 (a small number of these belong to row 16 or row 4 instead)\n",
20739+
"# upgradeHSTS_direct_direct ~800/828 (a small number of these belong to row 16 or row 4 instead)\n",
20740+
"\n",
20741+
"# Other: \n",
20742+
"# subresourceloadingCOEP_img_direct some noise (in either the new or the old Safari) 28/67; mostly cannot reproduce, could be due to some caching or timeout issuse?!\n",
20743+
"# 13 of them are 1x load in old version and 2x load in new version, the other 15 are the other way round"
20744+
]
20745+
},
20746+
{
20747+
"cell_type": "code",
20748+
"execution_count": 229,
2065420749
"id": "bdf73f36-3c44-426e-ba41-55c46dee9562",
2065520750
"metadata": {},
2065620751
"outputs": [
2065720752
{
2065820753
"data": {
2065920754
"application/vnd.jupyter.widget-view+json": {
20660-
"model_id": "1be7b09f91ab41df8725163ced9a866f",
20755+
"model_id": "4ea8d8f248d349b4a83ee4c3fee71baf",
2066120756
"version_major": 2,
2066220757
"version_minor": 0
2066320758
},
@@ -20677,6 +20772,70 @@
2067720772
"show_response_groups(test_name, browser_ids=browser_ids, relation=relation)"
2067820773
]
2067920774
},
20775+
{
20776+
"cell_type": "code",
20777+
"execution_count": 230,
20778+
"id": "9093f530-8575-434a-955b-9bee62f6db76",
20779+
"metadata": {},
20780+
"outputs": [
20781+
{
20782+
"data": {
20783+
"text/html": [
20784+
"<div>\n",
20785+
"<style scoped>\n",
20786+
" .dataframe tbody tr th:only-of-type {\n",
20787+
" vertical-align: middle;\n",
20788+
" }\n",
20789+
"\n",
20790+
" .dataframe tbody tr th {\n",
20791+
" vertical-align: top;\n",
20792+
" }\n",
20793+
"\n",
20794+
" .dataframe thead th {\n",
20795+
" text-align: right;\n",
20796+
" }\n",
20797+
"</style>\n",
20798+
"<table border=\"1\" class=\"dataframe\">\n",
20799+
" <thead>\n",
20800+
" <tr style=\"text-align: right;\">\n",
20801+
" <th></th>\n",
20802+
" <th>browser</th>\n",
20803+
" </tr>\n",
20804+
" <tr>\n",
20805+
" <th>outcome_str</th>\n",
20806+
" <th></th>\n",
20807+
" </tr>\n",
20808+
" </thead>\n",
20809+
" <tbody>\n",
20810+
" <tr>\n",
20811+
" <th>{'response.redirected': False}</th>\n",
20812+
" <td>[safari macOS 15.2 18.2 selenium real]</td>\n",
20813+
" </tr>\n",
20814+
" <tr>\n",
20815+
" <th>{'response.redirected': True}</th>\n",
20816+
" <td>[brave Android 11 1.62.165_shield intent real, brave Ubuntu 22.04 v1.62.156 (121.0.6167.139) selenium headless-new, brave Ubuntu 22.04 v1.73.101 (Chromium 131.0.6778.139) selenium headless-new, chrome Android 11 121.0.6167.180 intent real, chrome Ubuntu 22.04 120 selenium headless-new, chrome Ubuntu 22.04 121 selenium headless-new, chrome Ubuntu 22.04 122 selenium headless-new, chrome Ubuntu 22.04 131 selenium headless-new, chrome iPadOS 17.3.1 122.0.6261.89 intent real, firefox Ubuntu 22.04 121 selenium headless, firefox Ubuntu 22.04 122 selenium headless, firefox Ubuntu 22.04 123 selenium headless, firefox Ubuntu 22.04 133 selenium headless, firefox_beta Android 11 123.0b9 intent real, safari macOS 14.3.1 17.3.1 selenium real]</td>\n",
20817+
" </tr>\n",
20818+
" </tbody>\n",
20819+
"</table>\n",
20820+
"</div>"
20821+
],
20822+
"text/plain": [
20823+
" browser\n",
20824+
"outcome_str \n",
20825+
"{'response.redirected': False} [safari macOS 15.2 18.2 selenium real]\n",
20826+
"{'response.redirected': True} [brave Android 11 1.62.165_shield intent real, brave Ubuntu 22.04 v1.62.156 (121.0.6167.139) selenium headless-new, brave Ubuntu 22.04 v1.73.101 (Chromium 131.0.6778.139) selenium headless-new, chrome Android 11 121.0.6167.180 intent real, chrome Ubuntu 22.04 120 selenium headless-new, chrome Ubuntu 22.04 121 selenium headless-new, chrome Ubuntu 22.04 122 selenium headless-new, chrome Ubuntu 22.04 131 selenium headless-new, chrome iPadOS 17.3.1 122.0.6261.89 intent real, firefox Ubuntu 22.04 121 selenium headless, firefox Ubuntu 22.04 122 selenium headless, firefox Ubuntu 22.04 123 selenium headless, firefox Ubuntu 22.04 133 selenium headless, firefox_beta Android 11 123.0b9 intent real, safari macOS 14.3.1 17.3.1 selenium real]"
20827+
]
20828+
},
20829+
"metadata": {},
20830+
"output_type": "display_data"
20831+
}
20832+
],
20833+
"source": [
20834+
"test_id = \"upgradeHSTS_direct_direct_http_sub.headers.websec.saarland_https_sub.headers.websec.saarland_32127_parsing\"\n",
20835+
"with pd.option_context(\"display.max_colwidth\", None):\n",
20836+
" display(tree_df.loc[tree_df[\"test_id\"] == test_id].groupby(\"outcome_str\")[\"browser\"].unique().to_frame())"
20837+
]
20838+
},
2068020839
{
2068120840
"cell_type": "markdown",
2068220841
"id": "24413de8-42be-407f-a34e-03e202d6422f",
@@ -20688,6 +20847,7 @@
2068820847
"- Start: 14:53\n",
2068920848
"- End: 15:09\n",
2069020849
"- Time Taken: 16m\n",
20850+
"- Total of 286 diffs\n",
2069120851
"- UpgradeHSTS: 202+32=234 cases, various fixes (#16 in table), only remaining differences with Firefox (#22, different caching of code 300)\n",
2069220852
"- Framing: 39 cases, fix uppercase scheme (#11 in table)\n",
2069320853
"- Fullscreen: 9 cases, fix uppercase scheme (#11? not in table as only chrome supports PP header)\n",
@@ -20879,6 +21039,7 @@
2087921039
"- Start: 15:10\n",
2088021040
"- End: 15:30\n",
2088121041
"- Time: 20m\n",
21042+
"- Total of 294 diffs\n",
2088221043
"- Identical to Chrome (234+9+8+2; #16 HSTS fixes and #11 CSP uppercase scheme fix)\n",
2088321044
"- Additionally: referrer, 8 cases, 7 cases stricter HTTPS upgrades (window.open URLs automatically get upgraded) (related to #9), 1 case could be noise or only works the first time"
2088421045
]
@@ -21068,8 +21229,9 @@
2106821229
"- Start: 15:35\n",
2106921230
"- End: 16:15\n",
2107021231
"- Time: 40m\n",
21232+
"- Total of 167 diffs\n",
2107121233
"- Framing: 38 + 19, code 300 (#7, fixed), XFO whitspace (#15, fixed)\n",
21072-
"- SubresourceloadingCOPE: 6 (code 300, #7 fixed), 32 continuing random CORP caching (#35)\n",
21234+
"- SubresourceloadingCOEP: 6 (code 300, #7 fixed), 32 continuing random CORP caching (#35)\n",
2107321235
"- perfAPI/TAO: 2 (code 300, #7 fixed), 16 related to #29 (not fixed but changed, TODO update bug report?, entry is still with the old URL but requestStart is 0 even though it should not be 0)\n",
2107421236
"- fetch: 18 (code 300, #7 fixed)\n",
2107521237
"- access_window: 8x changed handling of extra \\n in headers, before such responses were downloaded (null) now they are rendered as plaintext (related to #3 and #37), probably known?\n",

0 commit comments

Comments
 (0)