serial: core: make sure RS485 cannot be enabled when it is not supported

Some uart drivers specify a rs485_config() function and then decide later
to disable RS485 support for some reason (e.g. imx and ar933).

In these cases userspace may be able to activate RS485 via TIOCSRS485
nevertheless, since in uart_set_rs485_config() an existing rs485_config()
function indicates that RS485 is supported.

Make sure that this is not longer possible by checking the uarts
rs485_supported.flags instead and bailing out if SER_RS485_ENABLED is not
set.

Furthermore instead of returning an empty structure return -ENOTTY if the
RS485 configuration is requested via TIOCGRS485 but RS485 is not supported.
This has a small impact on userspace visibility but it is consistent with
the -ENOTTY error for TIOCGRS485.

Fixes: e849145 ("serial: ar933x: Fill in rs485_supported")
Fixes: 55e18c6 ("serial: imx: Remove serial_rs485 sanitization")
Cc: Shawn Guo <[email protected]>
Cc: Sascha Hauer <[email protected]>
Cc:  <[email protected]>
Reviewed-by: Ilpo Järvinen <[email protected]>
Signed-off-by: Lino Sanfilippo <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Greg Kroah-Hartman <[email protected]>

Author: linosanfilippo-kunbus

drivers/tty/serial/serial_core.c

@@ -1474,7 +1474,7 @@ static int uart_set_rs485_config(struct tty_struct *tty, struct uart_port *port,
 	int ret;
 	unsigned long flags;
 
-	if (!port->rs485_config)
+	if (!(port->rs485_supported.flags & SER_RS485_ENABLED))
 		return -ENOTTY;
 
 	if (copy_from_user(&rs485, rs485_user, sizeof(*rs485_user)))