Allowing using ephemeral ports for testing with Screen

[drcraig]

The Screen class for testing currently hard-codes the port number to 3392. This makes it challenging to run a testsuite that tests multiple different nicegui apps in parallel. A solution for this could be to allow (or even default to) using ephemeral ports, by setting Screen.PORT to 0 and letting the operating system assign an ephemeral port.

I don't have a strong opinion on whether Screen should default to using an ephemeral port vs. that being an option. I could see allowing the port to be an init argument of Screen, and then bubbling that setting up through the screen fixture and its dependencies so that the user could set it. Perhaps the port number could be a fixture that the user could override.

For this to work, there would also need to be a change to ui.run, which check the falsiness of the port argument when deciding whether to use the port argument or default to 8080. This prevents passing a port of zero.

I'd be happy to help contribute this change.

[rodja]

Thanks for bringing this up @drcraig. You can already configure the port by simply calling Screen.PORT = 3930 or similar before your tests. And in native mode we already have code to find an open port:

nicegui/nicegui/native/native_mode.py

Lines 123 to 136 in 5bd08f9

	def find_open_port(start_port: int = 8000, end_port: int = 8999) -> int:
	"""Reliably find an open port in a given range.
	This function will actually try to open the port to ensure no firewall blocks it.
	This is better than, e.g., passing port=0 to uvicorn.
	"""
	for port in range(start_port, end_port + 1):
	try:
	with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
	s.bind(('localhost', port))
	return port
	except OSError:
	pass
	raise OSError('No open port found')

We could set Screen.PORT to None by default and assign an open port in Screen.__init__ if it is None. What do you think? Would you like to create a pull request?

[evnchn]

That would be great, if the screen fixture automatically finds an open port. Since that can make parallelizing tests more easy, and we can run our test suite faster. Most likely I'm going to hold off my plans to build a testing server from cheap second-hand Xeons until this gets done, then.

[drcraig]

Thanks for the reply @rodja .

You can already configure the port by simply calling Screen.PORT = 3930 or similar before your tests.

That's exactly what I ended up doing. I thought originally I would have to reimplement the fixtures that use the modified Screen class, but realized later that simply monkeypatching the Screen class just works. I created a wrapper plugin that seems to do the trick. (I opted to not use the find_open_port that nicegui ships because I prefer a port from the ephemeral range rather than the 8000s to minimize any risk of conflicts.)

import socket

from nicegui.testing.plugin import *  # noqa: F403
from nicegui.testing.screen import Screen


def get_ephemeral_port():
    """Get an ephemeral port that is not in use."""
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        sock.bind(("127.0.0.1", 0))
        return sock.getsockname()[1]


Screen.PORT = get_ephemeral_port()  # type: ignore

I can create a PR to make this behavior the default behavior. Let me know what your preference for port ranges are. I personally find it appealing to let the OS choose the port from the ephemeral range by setting the port argument to zero in socket.bind() rather than having to find my own port. But if you'd prefer it to use from the 8000s, we can do that, too. I'll start with the first implementation and you can let me know what you think.

[drcraig]

When making the change, I had to update a bunch of test helper functions to not use Screen.PORT and instead become factory fixtures that depended on the screen fixture and then get the port from the instance of Screen. I wonder how many nicegui users do the same thing, which could make this a breaking change.

[evnchn]

Perhaps you can PR so that we can see it for ourselves and assess it.

Note that we are a bit busy though, so it may take a while.

[rodja]

I think in the case of tests passing port 0 is a good solution. We do not need to fear firewalls blocking the port because it will only be accessed from the test-machine.