[Feature request] Gateway mode

[himekifee]

Will cgtproxy supports gateway mode like cgproxy have been? It is quite handy to deploy it on VPS and use the VPS to route client traffic. This would require gateway mode.

[black-desk]

I haven't test this case, let me check what is going wrong later.

[himekifee]

Thanks. I was just testing with the Zerotier custom route feature to enable cross-internet private IP access. Though the packet never entered v2ray-MARK. I guess there are some other rules accept that somewhere before cgtproxy had a chance to deal with that.

[black-desk]

I am not really familiar with netfliter things at all.

Let me note why package from bridge device are not being redirected to tproxy server at now.

Currently, cgtproxy will produce rules like:

table inet cgtproxy {
        set bypass {
                type ipv4_addr
                flags interval
                elements = { 127.0.0.0/8 }
        }

        set bypass6 {
                type ipv6_addr
                flags interval
                elements = { ::1 }
        }

        map cgroup-vmap {
                type cgroupsv2 : verdict
                elements = { "init.scope" : goto clash-meta-MARK,
                             "system.slice" : goto clash-meta-MARK,
                             "dev-hugepages.mount" : goto clash-meta-MARK,
                             "dev-mqueue.mount" : goto clash-meta-MARK,
                             "sys-kernel-debug.mount" : goto clash-meta-MARK,
                             "sys-kernel-tracing.mount" : goto clash-meta-MARK,
                             "sys-kernel-config.mount" : goto clash-meta-MARK,
                             "sys-fs-fuse-connections.mount" : goto clash-meta-MARK,
                             "machine.slice" : goto clash-meta-MARK,
                             "system.slice/system-getty.slice" : goto clash-meta-MARK,
                             "system.slice/system-modprobe.slice" : goto clash-meta-MARK,
                             "system.slice/system-systemd\x2dfsck.slice" : goto clash-meta-MARK,
                             "user.slice" : goto clash-meta-MARK,
                             "system.slice/dev-nvme0n1p2.swap" : goto clash-meta-MARK,
                             "system.slice/systemd-journald.service" : goto clash-meta-MARK,
                             "system.slice/home.mount" : goto clash-meta-MARK,
                             "system.slice/systemd-udevd.service" : goto clash-meta-MARK,
                             "system.slice/systemd-udevd.service/udev" : goto clash-meta-MARK,
                             "system.slice/boot-efi.mount" : goto clash-meta-MARK,
                             "proc-sys-fs-binfmt_misc.mount" : goto clash-meta-MARK,
                             "system.slice/systemd-timesyncd.service" : goto clash-meta-MARK,
                             "system.slice/avahi-daemon.service" : goto clash-meta-MARK,
                             "system.slice/blackdesk-tpl14g2-fix-led.service" : goto clash-meta-MARK,
                             "system.slice/dbus.service" : goto clash-meta-MARK,
                             "system.slice/low-memory-monitor.service" : goto clash-meta-MARK,
                             "system.slice/polkit.service" : goto clash-meta-MARK,
                             "system.slice/smartmontools.service" : goto clash-meta-MARK,
                             "system.slice/accounts-daemon.service" : goto clash-meta-MARK,
                             "system.slice/cron.service" : goto clash-meta-MARK,
                             "system.slice/switcheroo-control.service" : goto clash-meta-MARK,
                             "system.slice/systemd-logind.service" : goto clash-meta-MARK,
                             "system.slice/systemd-machined.service" : goto clash-meta-MARK,
                             "system.slice/udisks2.service" : goto clash-meta-MARK,
                             "system.slice/virtlockd.service" : goto clash-meta-MARK,
                             "system.slice/virtlogd.service" : goto clash-meta-MARK,
                             "system.slice/vmware-USBArbitrator.service" : goto clash-meta-MARK,
                             "system.slice/NetworkManager.service" : goto clash-meta-MARK,
                             "system.slice/wpa_supplicant.service" : goto clash-meta-MARK,
                             "system.slice/ModemManager.service" : goto clash-meta-MARK,
                             "system.slice/cups.service" : goto clash-meta-MARK,
                             "system.slice/ssh.service" : goto clash-meta-MARK,
                             "system.slice/gdm.service" : goto clash-meta-MARK,
                             "system.slice/rtkit-daemon.service" : goto clash-meta-MARK,
                             "system.slice/upower.service" : goto clash-meta-MARK,
                             "system.slice/colord.service" : goto clash-meta-MARK,
                             "system.slice/cups-browsed.service" : goto clash-meta-MARK,
                             "system.slice/vmware.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/init.scope" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/background.slice" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/dbus.socket" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/blackdesk-idle-on-battery.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/session-6.scope" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/fcitx.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/pipewire.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/filter-chain.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/wireplumber.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/pipewire-pulse.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/gnome-keyring-daemon.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/dbus.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/gvfs-daemon.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/user.slice" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/user.slice/podman-pause-4ca31300.scope" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/background.slice/tracker-miner-fs-3.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/xdg-document-portal.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/xdg-permission-store.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/gvfs-udisks2-volume-monitor.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/gvfs-mtp-volume-monitor.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/gvfs-gphoto2-volume-monitor.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome\x2dsession\x2dmanager.slice" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/gcr-ssh-agent.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/gnome-session-monitor.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/gvfs-goa-volume-monitor.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome\x2dsession\x2dmanager.slice/gnome-session-manager@gnome.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/gvfs-afc-volume-monitor.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.Shell@wayland.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-at\x2dspi\x2ddbus\x2dbus-9555.scope" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/evolution-source-registry.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/gvfs-metadata.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/evolution-calendar-factory.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/evolution-addressbook-factory.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.A11ySettings.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.Color.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.Datetime.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.Housekeeping.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.Keyboard.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.MediaKeys.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.Power.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.PrintNotifications.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.Rfkill.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.ScreensaverProxy.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.Sharing.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.Smartcard.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.Sound.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.UsbProtection.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.Wacom.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-org.gnome.Evolution\x2dalarm\x2dnotify-10055.scope" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-org.gnome.SettingsDaemon.DiskUtilityNotify-10042.scope" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-org.gnome.Software-10096.scope" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-remmina\x2dapplet-10087.scope" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/org.gnome.SettingsDaemon.XSettings.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-org.wezfurlong.wezterm@13586.service" : goto clash-meta-MARK,
                             "system.slice/bluetooth.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/dconf.service" : goto clash-meta-MARK,
                             "system.slice/system-systemd\x2dbacklight.slice" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/session.slice/xdg-desktop-portal.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/xdg-desktop-portal-gnome.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/xdg-desktop-portal-gtk.service" : goto clash-meta-MARK,
                             "system.slice/cgtproxy.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/gpg-agent.service" : goto clash-meta-MARK,
                             "system.slice/clash-meta.service" : return,
                             "system.slice/system-systemd\x2dcoredump.slice" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-org.wezfurlong.wezterm@6130.service" : goto clash-meta-MARK,
                             "system.slice/fwupd.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-busctl-283897.scope" : goto clash-meta-MARK,
                             "system.slice/pcscd.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-firefox\x2desr-322688.scope" : goto clash-meta-MARK,
                             "system.slice/org.deepin.linglong.PackageManager.service" : goto clash-meta-MARK,
                             "system.slice/org.deepin.linglong.SystemHelper.service" : goto clash-meta-MARK,
                             "user.slice/user-1000.slice/user@1000.service/app.slice/org.deepin.linglong.AppManager.service" : goto clash-meta-MARK }
        }

        map mark-vmap {
                type mark : verdict
                elements = { 0x00000bb8 : goto clash-meta }
        }

        map mark-dns-vmap {
                type mark : verdict
                elements = { 0x00000bb8 : goto clash-meta-DNS }
        }

        chain output-mangle {
                type route hook output priority mangle; policy accept;
                ct direction reply return
                ip daddr @bypass return
                ip6 daddr @bypass6 return
                meta l4proto != { tcp, udp } return
                socket cgroupv2 level 6 vmap @cgroup-vmap
                socket cgroupv2 level 5 vmap @cgroup-vmap
                socket cgroupv2 level 4 vmap @cgroup-vmap
                socket cgroupv2 level 3 vmap @cgroup-vmap
                socket cgroupv2 level 2 vmap @cgroup-vmap
                socket cgroupv2 level 1 vmap @cgroup-vmap
        }

        chain output-nat {
                type nat hook output priority dstnat; policy accept;
                meta mark vmap @mark-dns-vmap
        }

        chain prerouting {
                type filter hook prerouting priority mangle; policy accept;
                ip daddr @bypass return
                ip6 daddr @bypass6 return
                meta mark vmap @mark-vmap
        }

        chain clash-meta-MARK {
                meta mark set 0x00000bb8
        }

        chain clash-meta {
                meta l4proto { tcp, udp } tproxy to :7893
        }

        chain clash-meta-DNS {
                udp dport 53 dnat ip to 127.0.0.1:53
        }
}

check this:

chain output-mangle {
                type route hook output priority mangle; policy accept;
                ct direction reply return
                ip daddr @bypass return
                ip6 daddr @bypass6 return
                meta l4proto != { tcp, udp } return
                socket cgroupv2 level 6 vmap @cgroup-vmap
                socket cgroupv2 level 5 vmap @cgroup-vmap
                socket cgroupv2 level 4 vmap @cgroup-vmap
                socket cgroupv2 level 3 vmap @cgroup-vmap
                socket cgroupv2 level 2 vmap @cgroup-vmap
                socket cgroupv2 level 1 vmap @cgroup-vmap
        }

For package not send via bridge or something like that, socket cgroupv2 will not take effect at all.

It result that package not marked at all, which means a DIRECT.

[himekifee]

I see. My use case was using wireguard to proxy traffic to the central wireguard server and serve as a gateway. Though, as a kernel module, it does not really have a cgroup so that's probably why it doesn't work. Close for now.

[black-desk]

I think it is fine to have a default tproxy target when traffic doesn't have a cgroup, which is the "gateway mode" you wanted. I am finding ways to implement this feature.

This might related to #79