Skip to content

Commit ae5d4eb

Browse files
committed
[Cocoa] Add support for overriding DNS resolution for WPT
https://bugs.webkit.org/show_bug.cgi?id=261038 rdar://82927182 Reviewed by Alex Christensen. Patch by Matthew Finkel and Anne van Kesteren with minor modifications by Per Arne. Some ports don't have support for resolving arbitrary domains, but web-platform-tests rely on this capability. Currently, the Apple ports only use localhost, and 127.0.0.1 as a workaround where it's necessary, but this affects our test coverage. This patch adds a very basic DNS resolver and introduces a new command-line argument for run-webkit-tests; this is disabled by default. In 254273@main, glib ports gained support for resolving arbitrary domains, and this moves Apple ports closer to supporting that. We expect to enable this by default, along with rebaselining, in a later commit. Previous attempts were made in https://bugs.webkit.org/show_bug.cgi?id=245294, and 264076@main introduced part of that behavior for a different reason. In some situations, name resolutions falls back on that SPI, as well. The basic DNS resolver only resolves A records for names ending with "localhost" and for the host names defines by the port's localhost_aliases. The server always returns 127.0.0.1 as the answer for those hosts. All other queries return NXDomain. This resolver depends on the dnslib Python library. On Apple ports, the iOS Simulator and Mac use different implementations. The iOS simulator part is in the Network process because we need to define the resolver in-process otherwise it won't be used. The way the resolver is configured on Mac isn't supported on the simulator. On Mac, we use SPI and a Network Extensions policy so this works when a VPN is enabled. We do this in WebKitTestRunner so we don't need to add additional entitlements and weaken the sandbox for the network process. * Source/WebCore/PAL/pal/spi/cocoa/NetworkSPI.h: * Source/WebKit/NetworkProcess/cocoa/NetworkProcessCocoa.mm: (WebKit::NetworkProcess::platformInitializeNetworkProcessCocoa): * Tools/Scripts/webkitpy/__init__.py: * Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py: (parse_args): * Tools/Scripts/webkitpy/layout_tests/servers/basic_dns_server.py: Added. (Resolver): (Resolver.__init__): (Resolver.resolve): * Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_server.py: (WebPlatformTestServer.__init__): (WebPlatformTestServer._spawn_process): (WebPlatformTestServer._stop_running_server): * Tools/Scripts/webkitpy/port/apple.py: (ApplePort.__init__): * Tools/Scripts/webkitpy/port/driver.py: (Driver.cmd_line): * Tools/Scripts/webkitpy/port/test.py: * Tools/WebKitTestRunner/Configurations/WebKitTestRunner-internal.entitlements: * Tools/WebKitTestRunner/Options.cpp: (WTR::handleOptionLocalDNSResolver): (WTR::OptionsHandler::OptionsHandler): * Tools/WebKitTestRunner/Options.h: * Tools/WebKitTestRunner/TestController.h: * Tools/WebKitTestRunner/WebKitTestRunner.xcodeproj/project.pbxproj: * Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm: (WTR::TestController::cocoaPlatformInitialize): (WTR::TestController::cocoaDNSInitialize): Canonical link: https://commits.webkit.org/300026@main
1 parent 6160475 commit ae5d4eb

File tree

15 files changed

+252
-10
lines changed

15 files changed

+252
-10
lines changed

Source/WebCore/PAL/pal/spi/cocoa/NetworkSPI.h

Lines changed: 54 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/*
2-
* Copyright (C) 2021-2023 Apple Inc. All rights reserved.
2+
* Copyright (C) 2021-2025 Apple Inc. All rights reserved.
33
*
44
* Redistribution and use in source and binary forms, with or without
55
* modification, are permitted provided that the following conditions
@@ -33,6 +33,11 @@ DECLARE_SYSTEM_HEADER
3333

3434
#import <nw/private.h>
3535

36+
#if PLATFORM(MAC) && defined(__OBJC__)
37+
// Only needed for running tests.
38+
#import <NetworkExtension/NEPolicySession.h>
39+
#endif
40+
3641
#else
3742

3843
WTF_EXTERN_C_BEGIN
@@ -77,4 +82,52 @@ void nw_webtransport_options_set_allow_joining_before_ready(nw_protocol_options_
7782

7883
WTF_EXTERN_C_END
7984

85+
// ------------------------------------------------------------
86+
// The following declarations are only needed for running tests.
87+
88+
WTF_EXTERN_C_BEGIN
89+
90+
typedef enum {
91+
nw_resolver_protocol_dns53 = 0,
92+
} nw_resolver_protocol_t;
93+
94+
typedef enum {
95+
nw_resolver_class_designated_direct = 2,
96+
} nw_resolver_class_t;
97+
98+
nw_resolver_config_t nw_resolver_config_create(void);
99+
void nw_resolver_config_set_protocol(nw_resolver_config_t, nw_resolver_protocol_t);
100+
void nw_resolver_config_set_class(nw_resolver_config_t, nw_resolver_class_t);
101+
void nw_resolver_config_add_match_domain(nw_resolver_config_t, const char *);
102+
void nw_resolver_config_add_name_server(nw_resolver_config_t, const char *name_server);
103+
void nw_resolver_config_set_identifier(nw_resolver_config_t, const uuid_t identifier);
104+
bool nw_resolver_config_publish(nw_resolver_config_t);
105+
106+
WTF_EXTERN_C_END
107+
108+
#if defined(__OBJC__)
109+
typedef NS_ENUM(NSInteger, NEPolicySessionPriority) {
110+
NEPolicySessionPriorityHigh = 300,
111+
};
112+
113+
@interface NEPolicyCondition : NSObject
114+
+ (NEPolicyCondition *)domain:(NSString *)domain;
115+
@end
116+
117+
@interface NEPolicyResult : NSObject
118+
+ (NEPolicyResult *)netAgentUUID:(NSUUID *)agentUUID;
119+
@end
120+
121+
@interface NEPolicy : NSObject
122+
- (instancetype)initWithOrder:(uint32_t)order result:(NEPolicyResult *)result conditions:(NSArray<NEPolicyCondition *> *)conditions;
123+
@end
124+
125+
@interface NEPolicySession : NSObject
126+
@property NEPolicySessionPriority priority;
127+
- (NSUInteger)addPolicy:(NEPolicy *)policy;
128+
- (BOOL)apply;
129+
@end
130+
#endif // defined(__OBJC__)
131+
// ------------------------------------------------------------
132+
80133
#endif // USE(APPLE_INTERNAL_SDK)

Source/WebKit/NetworkProcess/cocoa/NetworkProcessCocoa.mm

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/*
2-
* Copyright (C) 2014-2018 Apple Inc. All rights reserved.
2+
* Copyright (C) 2014-2025 Apple Inc. All rights reserved.
33
*
44
* Redistribution and use in source and binary forms, with or without
55
* modification, are permitted provided that the following conditions
@@ -60,6 +60,8 @@
6060
#import "WKProcessExtension.h"
6161
#endif
6262

63+
#import <pal/spi/cocoa/NetworkSPI.h>
64+
6365
namespace WebKit {
6466

6567
static void initializeNetworkSettings()
@@ -118,6 +120,18 @@ static void initializeNetworkSettings()
118120
#endif
119121
m_enableModernDownloadProgress = parameters.enableModernDownloadProgress;
120122

123+
#if PLATFORM(IOS_SIMULATOR)
124+
// See TestController::cocoaPlatformInitialize for supporting a local DNS resolver on Mac.
125+
if (parameters.localhostAliasesForTesting.contains("web-platform.test"_s)) {
126+
nw_resolver_config_t resolverConfig = nw_resolver_config_create();
127+
nw_resolver_config_set_protocol(resolverConfig, nw_resolver_protocol_dns53);
128+
nw_resolver_config_set_class(resolverConfig, nw_resolver_class_designated_direct);
129+
nw_resolver_config_add_name_server(resolverConfig, "127.0.0.1:8053");
130+
nw_resolver_config_add_match_domain(resolverConfig, "test");
131+
nw_privacy_context_require_encrypted_name_resolution(NW_DEFAULT_PRIVACY_CONTEXT, true, resolverConfig);
132+
}
133+
#endif
134+
121135
increaseFileDescriptorLimit();
122136
}
123137

Tools/Scripts/webkitpy/__init__.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# Copyright (C) 2008-2020 Andrey Petrov and contributors.
2-
# Copyright (C) 2023 Apple Inc. All rights reserved.
2+
# Copyright (C) 2023-2025 Apple Inc. All rights reserved.
33
#
44
# Redistribution and use in source and binary forms, with or without
55
# modification, are permitted provided that the following conditions are
@@ -78,6 +78,7 @@
7878
AutoInstall.register(Package('configparser', Version(4, 0, 2), implicit_deps=['pyparsing'], aliases=['backports.configparser']))
7979
AutoInstall.register(Package('contextlib2', Version(0, 6, 0)))
8080
AutoInstall.register(Package('coverage', Version(7, 6, 1), wheel=True))
81+
AutoInstall.register(Package('dnslib', Version(0, 9, 26)))
8182
AutoInstall.register(Package('funcsigs', Version(1, 0, 2)))
8283
AutoInstall.register(Package('html5lib', Version(1, 1)))
8384
AutoInstall.register(Package('iniconfig', Version(1, 1, 1)))

Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Copyright (C) 2010 Google Inc. All rights reserved.
22
# Copyright (C) 2010 Gabor Rapcsanyi ([email protected]), University of Szeged
3-
# Copyright (C) 2011, 2016, 2019 Apple Inc. All rights reserved.
3+
# Copyright (C) 2011-2025 Apple Inc. All rights reserved.
44
#
55
# Redistribution and use in source and binary forms, with or without
66
# modification, are permitted provided that the following conditions are
@@ -377,6 +377,7 @@ def parse_args(args):
377377
option_group_definitions.append(("Web Platform Test Server Options", [
378378
optparse.make_option("--disable-wpt-hostname-aliases", action="store_true", default=False, help="Disable running tests from WPT against the web-platform.test domain, if the port supports it."),
379379
optparse.make_option("--wptserver-doc-root", type="string", help=("Set web platform server document root, relative to LayoutTests directory")),
380+
optparse.make_option("--local-dns-resolver", action="store_true", default=False, help="Run and use a local DNS resolver, if the port supports it.")
380381
]))
381382

382383
option_group_definitions.append(('Upload Options', upload_options()))
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Copyright (C) 2023-2025 Apple Inc. All rights reserved.
2+
#
3+
# Redistribution and use in source and binary forms, with or without
4+
# modification, are permitted provided that the following conditions
5+
# are met:
6+
# 1. Redistributions of source code must retain the above copyright
7+
# notice, this list of conditions and the following disclaimer.
8+
# 2. Redistributions in binary form must reproduce the above copyright
9+
# notice, this list of conditions and the following disclaimer in the
10+
# documentation and/or other materials provided with the distribution.
11+
#
12+
# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS "AS IS" AND
13+
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
14+
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
15+
# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR
16+
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
17+
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
18+
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
19+
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
20+
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
21+
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
22+
23+
from dnslib import CLASS, QTYPE, RR, RCODE
24+
from dnslib.server import BaseResolver, DNSServer
25+
import logging
26+
from threading import Thread
27+
28+
_log = logging.getLogger(__name__)
29+
30+
31+
class Resolver(BaseResolver):
32+
def __init__(self, allowed_hosts=[]):
33+
super().__init__()
34+
self.hosts = list(map(lambda x: x if x.endswith(".") else x + ".", allowed_hosts))
35+
_log.debug("Initializing Resolver with hosts: {}".format(self.hosts))
36+
37+
def resolve(self, request, handler):
38+
question = request.q
39+
reply = request.reply()
40+
_log.debug("Received request for {}".format(question.qname))
41+
if question.qtype == QTYPE.A and (question.qname in self.hosts or question.qname.matchSuffix("localhost")):
42+
reply.add_answer(*RR.fromZone("{} 3600 A 127.0.0.1".format(question.qname)))
43+
else:
44+
reply.header.rcode = getattr(RCODE, "NXDOMAIN")
45+
return reply
46+
47+
48+
if __name__ == "__main__":
49+
# This script is not intended to be run on its own, and should only be used for testing purposes.
50+
server = DNSServer(Resolver(["site.example"]), port=8053, address="127.0.0.1")
51+
server.start_thread()
52+
try:
53+
Thread.join()
54+
except Exception:
55+
pass
56+
finally:
57+
server.stop()

Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_server.py

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright (c) 2014, Canon Inc. All rights reserved.
1+
# Copyright (c) 2014 Canon Inc. All rights reserved.
22
# Redistribution and use in source and binary forms, with or without
33
# modification, are permitted provided that the following conditions
44
# are met:
@@ -27,6 +27,10 @@
2727
import time
2828

2929
from webkitpy.layout_tests.servers import http_server_base
30+
try:
31+
from webkitpy.layout_tests.servers.basic_dns_server import DNSServer, Resolver
32+
except ImportError:
33+
print("Error importing DNSServer, Resolver")
3034

3135
_log = logging.getLogger(__name__)
3236

@@ -127,6 +131,11 @@ def __init__(self, port_obj, name, pidfile=None):
127131
self._output_log_path = None
128132
self._wsout = None
129133
self._process = None
134+
self._dns_server = None
135+
if port_obj.supports_localhost_aliases and port_obj.get_option('local_dns_resolver') and not port_obj.get_option('disable_wpt_hostname_aliases'):
136+
self._dns_server = DNSServer(Resolver(
137+
allowed_hosts=port_obj.localhost_aliases()), port=8053, address="127.0.0.1")
138+
130139
self._pid_file = pidfile
131140
if not self._pid_file:
132141
self._pid_file = self._filesystem.join(self._runtime_path, '%s.pid' % self._name)
@@ -178,6 +187,9 @@ def _spawn_process(self):
178187
self._process = self._executive.popen(self._start_cmd, cwd=self._doc_root_path, shell=False, stdin=self._executive.PIPE, stdout=self._wsout, stderr=self._wsout)
179188
self._filesystem.write_text_file(self._pid_file, str(self._process.pid))
180189

190+
if self._dns_server:
191+
self._dns_server.start_thread()
192+
181193
# Wait a second for the server to actually start so that tests do not start until server is running.
182194
time.sleep(1)
183195

@@ -202,6 +214,9 @@ def _stop_running_server(self):
202214
self._wsout.close()
203215
self._wsout = None
204216

217+
if self._dns_server and hasattr(self._dns_server, "thread") and self._dns_server.isAlive():
218+
self._dns_server.stop()
219+
205220
if self._process is not None:
206221
self._process.poll()
207222

Tools/Scripts/webkitpy/port/apple.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@ def __init__(self, host, port_name, **kwargs):
8383

8484
port_name = port_name.replace('-wk2', '')
8585
self._version = self._strip_port_name_prefix(port_name)
86+
self.supports_localhost_aliases = False
8687

8788
def setup_test_run(self, device_type=None):
8889
self._crash_logs_to_skip_for_host[self.host] = self.host.filesystem.files_under(self.path_to_crash_logs())

Tools/Scripts/webkitpy/port/driver.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# Copyright (C) 2011 Google Inc. All rights reserved.
2-
# Copyright (c) 2015-2019 Apple Inc. All rights reserved.
2+
# Copyright (c) 2015-2025 Apple Inc. All rights reserved.
33
#
44
# Redistribution and use in source and binary forms, with or without
55
# modification, are permitted provided that the following conditions are
@@ -582,6 +582,8 @@ def cmd_line(self, pixel_tests, per_test_args):
582582
cmd.append('--show-window')
583583
if self._port.get_option('accessibility_isolated_tree'):
584584
cmd.append('--accessibility-isolated-tree')
585+
if self._port.get_option('local_dns_resolver'):
586+
cmd.append('--local-dns-resolver')
585587

586588
for allowed_host in self._port.allowed_hosts():
587589
cmd.append('--allowed-host')

Tools/Scripts/webkitpy/port/test.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,8 @@
3030

3131
from webkitcorepy import string_utils
3232

33-
from webkitpy.port import Port, Driver, DriverOutput
33+
from webkitpy.port.base import Port
34+
from webkitpy.port.driver import Driver, DriverOutput
3435
from webkitpy.layout_tests.models.test_configuration import TestConfiguration
3536
from webkitpy.common.system.crashlogs import CrashLogs
3637
from webkitpy.common.version_name_map import PUBLIC_TABLE, VersionNameMap

Tools/WebKitTestRunner/Configurations/WebKitTestRunner-internal.entitlements

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@
88
</array>
99
<key>com.apple.private.webkit.adattributiond.testing</key>
1010
<true/>
11+
<key>com.apple.private.nehelper.privileged</key>
12+
<true/>
13+
<key>com.apple.private.neagent</key>
14+
<true/>
15+
<key>com.apple.networkd.persistent_interface</key>
16+
<true/>
1117
<key>com.apple.security.temporary-exception.sbpl</key>
1218
<array>
1319
<string>(allow mach-issue-extension (require-all (extension-class &quot;com.apple.webkit.extension.mach&quot;)))</string>

0 commit comments

Comments
 (0)