Merge pull request #80 from Jakeler/tcp-socket

TCP socket server option

Author: Jakeler

README.md

@@ -183,15 +183,13 @@ As you can see, here the read/notify UUID is `00000006-af0e-4c28-95a4-4509fd91e0
 The `ble-serial` tool itself has a few more options:
 ```console
 $ ble_serial -h
-usage: __main__.py [-h] [-v] -d DEVICE [-t {public,random}] [-i ADAPTER] [-m MTU] [-w WRITE_UUID] [-l FILENAME] [-b]
-                   [-p PORT] [-r READ_UUID]
+usage: __main__.py [-h] [-v] [-t SEC] [-i ADAPTER] [-m MTU] [-d DEVICE] [-a {public,random}] [-s SERVICE_UUID] [-w WRITE_UUID] [-r READ_UUID] [--permit {ro,rw,wo}] [-l FILENAME] [-b] [-p PORT] [--expose-tcp-host TCP_HOST] [--expose-tcp-port TCP_PORT]
 
 Create virtual serial ports from BLE devices.
 
 options:
   -h, --help            show this help message and exit
   -v, --verbose         Increase verbosity, can be specified multiple times for connection/DBus debugging (default: 0)
-  -p PORT, --port PORT  Symlink to virtual serial port (default: /tmp/ttyBLE)
 
 connection parameters:
   -t SEC, --timeout SEC
@@ -212,13 +210,8 @@ device parameters:
   -r READ_UUID, --read-uuid READ_UUID
                         The GATT characteristic to subscribe to notifications to read the serial data (default: None)
   --permit {ro,rw,wo}   Restrict transfer direction on bluetooth: read only (ro), read+write (rw), write only (wo) (default: rw)
-
-logging options:
-  -l FILENAME, --log FILENAME
-                        Enable optional logging of all bluetooth traffic to file (default: None)
-  -b, --binary          Log data as raw binary, disable transformation to hex. Works only in combination with -l (default: False)
-
 ```
+
 In any case it needs to know which device to connect, the simple and most reliable way to specify this is by device address/id:
 ```console
 $ ble-serial -d 20:91:48:4c:4c:54
@@ -232,7 +225,11 @@ $ ble-serial -d 20:91:48:4c:4c:54
 
 ```
 This log shows a successful start on Linux, the virtual serial port was opened on `/dev/pts/8`, the number at the end changes, depending on how many pseudo terminals are already open on the system. It uses the same mechanism on macOS, just the path is slightly different, in the format `/dev/ttys000`.
-In addition it automatically creates a symlink to `/tmp/ttyBLE`, so you can easily access it always on the same file, the default can be changed with the `-p`/`--port` option.
+In addition it automatically creates a symlink to `/tmp/ttyBLE`, so you can easily access it always on the same file, the default can be changed with:
+```
+serial port parameters:
+  -p PORT, --port PORT  Symlink to virtual serial port (default: /tmp/ttyBLE)
+```
 
 On Windows it uses the port pair created in the setup described above, this does not dynamically change and endpoint is always `COM9` if you use the default script.
 
@@ -273,6 +270,13 @@ This works also the other way around with `wo` = write only.
 
 ### Logging
 There is an option to log all traffic on the link to a text file:
+```
+logging options:
+  -l FILENAME, --log FILENAME
+                        Enable optional logging of all bluetooth traffic to file (default: None)
+  -b, --binary          Log data as raw binary, disable transformation to hex. Works only in combination with -l (default: False)
+```
+
 ```console
 $ ble-serial -d 20:91:48:4c:4c:54 -l demo.txt
 ...
@@ -281,7 +285,7 @@ $ cat demo.txt
 2019-12-09 21:15:53.491681 -> BLE-IN: b0 b0 b0 b0 b0 b0 3b b0 b0 b0 ba b0 0d 8a
 2019-12-09 21:15:53.999795 -> BLE-IN: b0 b0 b0 b0 b0 b0 3b b0 b0 b0 ba b0 0d 8a
 ```
-Per default it is transformed to hex bytes, use `-b`/`--binary` to log raw data.
+Per default it is transformed to hex bytes, use `-b`/`--binary` to log raw data, useful if your input is already ASCII etc.
 
 You can use `-v` to increase the log verbosity to DEBUG:
 ```console
@@ -306,10 +310,49 @@ It also helps with figuring out how characteristics are selected:
 Always try the verbose option if something is not working properly.
 
 ## Advanced Usage
+### TCP socket server
+Instead of the serial port emulation there is a also builtin raw tcp server since version 2.7:
+```
+network options:
+  --expose-tcp-host TCP_HOST
+                        Network interface for the server listen on (default: 127.0.0.1)
+  --expose-tcp-port TCP_PORT
+                        Port to listen on, disables local serial port and enables TCP server if specified (default: None)
+```
+This is only activated if TCP_PORT is set. Also it removes the dependency to com0com or other drivers.
+
+The server is listening on localhost per default, therefore only reachable from apps running on the same machine. Other interfaces or `0.0.0.0` for all interfaces can be specified with TCP_HOST.
+Security is to consider though, this is plain TCP without encryption or authentication. Only recommended on a separate local network, otherwise stay with the default/localhost.
+
+Note: this is limited to one concurrent connection, it will reject all connection attempts if there is already a client connected and emit a warning, example:
+```console
+$ ble-serial -d 20:91:48:4C:4C:54 --expose-tcp-port 4002 -v
+[...]
+19:30:11.650 | INFO | main.py: Running main loop!
+19:30:11.650 | INFO | tcp_socket.py: TCP server started
+19:30:11.650 | DEBUG | tcp_socket.py: Listening on <asyncio.TransportSocket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 4002)>
+[...]
+19:30:13.618 | INFO | tcp_socket.py: New TCP peer connected: ('127.0.0.1', 49104)
+19:30:13.787 | DEBUG | ble_interface.py: Received notify from 0000ffe1-0000-1000-8000-00805f9b34fb (Handle: 17): Vendor specific: bytearray(b'\xb0\xb0\xb0\xb01\xb6;\xb0\xb0\xb0\xba\xb0\r\x8a')
+19:30:13.787 | DEBUG | tcp_socket.py: Sending: bytearray(b'\xb0\xb0\xb0\xb01\xb6;\xb0\xb0\xb0\xba\xb0\r\x8a')
+[...]
+19:30:26.726 | INFO | tcp_socket.py: New TCP peer connected: ('127.0.0.1', 56172)
+19:30:26.726 | WARNING | tcp_socket.py: More than one connection is not allowed, closing
+```
+
+Now there a various ways to connect to it. 
+#### Linux and macOS
+- Very simple option: `netcat 127.0.0.1 4002` or `telnet 127.0.0.1 4002`
+- More powerful: `socat -dd tcp:localhost:4002 -`, can forward data to many modules, not only stdin/stdout.
+- Custom apps are easy to make with tcp too
+#### Windows
+- Graphical: Putty, just put in the IP+port and select Other - Raw as connection type.
+- Terminal: netcat/telnet/socat can be installed separately
+
 ### Multi device connection
 It is possible to connect several devices to a host simultaneously. Limiting factor is only the bluetooth baseband layer, which uses a Active Member Address (AMA, 3 bit). From these 8 possible values address zero is always occupied by the host, so it can be connected to (up to) 7 devices at the same time.
 
-There is no special mode, ble-serial can be just stated multiple times with different parameters. Following are some tips, showing how to do this in practice.
+There is no special mode, ble-serial can be just started multiple times with different parameters. Following are some tips, showing how to do this in practice.
 
 #### Linux and macOS
 Common shells (bash, zsh, fish) have a useful background job feature. Add the `-p` (port) option to make sure every instance has a unique path. Also you probably want to redirect the log output to keep it separate. Resulting command lines could look like:

ble_serial/cli.py

@@ -7,9 +7,6 @@ def parse_args():
 
     parser.add_argument('-v', '--verbose', dest='verbose', action='count', default=0,
         help='Increase verbosity, can be specified multiple times for connection/DBus debugging')
-    parser.add_argument('-p', '--port', dest='port', required=False, default=DEFAULT_PORT,
-        help=DEFAULT_PORT_MSG)
-
 
     con_group = parser.add_argument_group('connection parameters')
     con_group.add_argument('-t', '--timeout', dest='timeout', required=False, default=5.0, type=float, metavar='SEC',
@@ -40,6 +37,16 @@ def parse_args():
     log_group.add_argument('-b', '--binary', dest='binlog', required=False, action='store_true',
         help='Log data as raw binary, disable transformation to hex. Works only in combination with -l')
 
+    uart_group = parser.add_argument_group('serial port parameters')
+    uart_group.add_argument('-p', '--port', dest='port', required=False, default=DEFAULT_PORT,
+        help=DEFAULT_PORT_MSG)
+
+    net_group = parser.add_argument_group('network options')
+    net_group.add_argument('--expose-tcp-host', dest='tcp_host', required=False, default='127.0.0.1',
+        help='Network interface for the server listen on')
+    net_group.add_argument('--expose-tcp-port', dest='tcp_port', required=False, default=None, type=int,
+        help='Port to listen on, disables local serial port and enables TCP server if specified')
+
     args = parser.parse_args()
 
     if not args.device and not args.service_uuid:

ble_serial/main.py

@@ -1,6 +1,7 @@
 import logging, asyncio
 from bleak.exc import BleakError
 from ble_serial import platform_uart as UART
+from ble_serial.ports.tcp_socket import TCP_Socket
 from ble_serial.bluetooth.ble_interface import BLE_interface
 from ble_serial.log.fs_log import FS_log, Direction
 from ble_serial.log.console_log import setup_logger
@@ -24,8 +25,13 @@ async def _run(self):
         loop = asyncio.get_event_loop()
         loop.set_exception_handler(self.excp_handler)
         try:
-            self.uart = UART(args.port, loop, args.mtu)
+            if args.tcp_port:
+                self.uart = TCP_Socket(args.tcp_host, args.tcp_port, args.mtu)
+            else:
+                self.uart = UART(args.port, loop, args.mtu)
+
             self.bt = BLE_interface(args.adapter, args.service_uuid)
+
             if args.filename:
                 self.log = FS_log(args.filename, args.binlog)
                 self.bt.set_receiver(self.log.middleware(Direction.BLE_IN, self.uart.queue_write))

ble_serial/ports/tcp_socket.py

@@ -0,0 +1,68 @@
+from ble_serial.ports.interface import ISerial
+import asyncio, logging
+
+
+class TCP_Socket(ISerial):
+    def __init__(self, host: str, port: int, mtu: int):
+        logging.debug(f'{host=}:{port=}, {mtu=}')
+        self.host = host
+        self.port = port
+        self.mtu = mtu
+        self.connected = False
+
+    def set_receiver(self, callback):
+        self._cb = callback
+
+    def queue_write(self, value: bytes):
+        if self.connected:
+            logging.debug(f'Sending: {value}')
+            self.writer.write(value)
+        else:
+            logging.debug(f'No client connected, dropping data...')
+
+    def handle_connect(self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter):
+        logging.info(f'New TCP peer connected: {writer.get_extra_info("peername")}')
+        if self.connected:
+            logging.warning('More than one connection is not allowed, closing')
+            writer.close()
+            return
+        self.writer = writer
+        self.reader = reader
+        self.connected = True
+
+
+    async def run_loop(self):
+        server =  await asyncio.start_server(
+            self.handle_connect, self.host, self.port)
+
+        logging.info('TCP server started' if server.is_serving() else 'TCP server failed')
+        for sock in server.sockets:
+            logging.debug(f'Listening on {sock}')
+
+        while True:
+            if self.connected:
+                try:
+                    data = await self.reader.read(self.mtu)
+                except OSError as ose:
+                    logging.warning(f'Client disconnected: {ose}')
+                    self.connected = False
+                    continue
+
+                if len(data) > 0:
+                    logging.debug(f'Received {data}')
+                    self._cb(data)
+                else:
+                    logging.warning('Client disconnected (EOF)')
+                    self.connected = False
+            else:
+                await asyncio.sleep(3)
+                logging.debug('Waiting for client connection')
+
+    def start(self):
+        pass
+
+    def stop_loop(self):
+        pass
+
+    def remove(self):
+        pass