could not add secure port example.test:50051 to server, not sure why

[omarluq]

Hello!

I'm trying to set up anycable for a Rails app dev environment that runs https with puma-dev but Im having issues with starting the rpc server specifically with setting up tls using the private CA

bundle exec anycable 
                                                                                                                                                                                                                 [bbed45ecc1]
Anyway Config was loaded before Rails. Activating Anyway Config Rails plugins now.
NOTE: Already loaded configs were provisioned without Rails-specific sources.

⛔️ WARNING: AnyCable RPC pool size (30) is greater than DB pool size (5)
Please, consider adjusting the database pool size to avoid connection wait times and increase throughput of your RPC server

Starting AnyCable RPC server (pid: 16718)
AnyCable version: 1.4.3 (proto_version: v1)
Serving Rails application from ./config/environment.rb
gRPC version: 1.59.0
E1130 21:33:34.264502000 8160059072 ssl_transport_security.cc:776]     Invalid cert chain file.
E1130 21:33:34.264523000 8160059072 ssl_security_connector.cc:240]     Handshaker factory creation failed with TSI_INVALID_ARGUMENT.
E1130 21:33:34.265408000 8160059072 chttp2_server.cc:1059]             UNKNOWN:Unable to create secure server with credentials of type Ssl {created_time:"2023-11-30T21:33:34.264528-06:00"}
could not add secure port example.test:50051 to server, not sure why

in my app I have the following config:

• In the main app

  config/anycable.yml

  default: &default
    access_logs_disabled: false
    log_grpc: false
    jwt_id_key: <%= ENV.fetch('ANYCABLE_JWT_ID_KEY', 'jwt-id-key') %>
    rpc_host: "example.test:50051"
    broadcast_adapter: redis
    redis_channel: "__anycable__"
    rpc_tls_cert: <%= ENV.fetch('ANYCABLE_CERT', '~/Library/Application\ Support/io.puma.dev/cert.pem') %>
    rpc_tls_key: <%= ENV.fetch('ANYCABLE_CERT_KEY', '~/Library/Application\ Support/io.puma.dev/key.pem') %>
    enable_rpc_tls: <%= ENV.fetch('ANYCABLE_ENABLE_RPC_TLS', true) %>
  
  
  
  development:
    <<: *default
    log_grpc: true
    embedded: false
  
  test:
    <<: *default
    embedded: true
  
  production:
    <<: *default

  config/cable.yml

  development:
    adapter: <%= ENV.fetch("ACTION_CABLE_ADAPTER", "any_cable") %>
  
  test:
    adapter: test
  
  production:
    adapter: <%= ENV.fetch("ACTION_CABLE_ADAPTER", "any_cable") %>

  config/initializers/anycable.rb

  AnyCable::Rails::Rack.middleware.use Warden::Manager do |config|
    Devise.warden_config = config
  end

  cable.js

  import { createCable } from "@anycable/web";
  
  const anyCableBaseConfig = { protocol: 'actioncable-v1-ext-json' };
  const anyCableDevConfig = { ...anyCableBaseConfig, debug: true, logLevel: 'debug', protocolOptions: { pongs: true } };
  const anyCableConfig = ['test', 'development'].includes(process.env.NODE_ENV) ? anyCableDevConfig : anyCableBaseConfig;
  export default createCable('wss://example.test/cable',anyCableConfig);

  hotwired_setup.js

  import { start } from '@anycable/turbo-stream';
  import cable from './cable';
  
  start(cable);

• In a docker compose that runs services connected to main app

    version: 'x.x'
    services:
      anycable_go:
        image: anycable/anycable-go:1.4.6
        environment:
              ANYCABLE_PORT: 8085
              REDIS_URL: redis://redis:6379/0
              ANYCABLE_HOST: 'example.test'
              ANYCABLE_JWT_ID_KEY: jwt-id-key
              ANYCABLE_TURBO_RAILS_KEY: s3cЯeT
              ANYCABLE_RPC_HOST: 'example.test:50051'
              ANYCABLE_BROADCAST_ADAPTER: redisx
              ANYCABLE_PUBSUB: redis
              ANYCABLE_BROKER: memory
              ANYCABLE_DEBUG: ${DEBUG:-1}
              ANYCABLE_SSL_CERT: /self_signed/anycable-cert.pem
              ANYCABLE_SSL_KEY: /self_signed/anycable-key.pem
              ANYCABLE_RPC_ENABLE_TLS: true
        depends_on:
              - redis
              - anycable_go_certs
        ports:
              - '8085:8085'
        volumes:
              - certs:/self_signed
      anycable_go_certs:
        image: alpine
        volumes:
              - certs:/certs
              - ~/Library/Application\ Support/io.puma.dev/:/source_certs
        command: >
              sh -c "cp ~/Library/Application\ Support/io.puma.dev/cert.pem /certs/anycable-cert.pem &&
                    cp ~/Library/Application\ Support/io.puma.dev/key.pem /certs/anycable-key.pem"
    volumes:
      certs:
      name: anycable_certs

  anycable-go seems to be working perfectly fine!

  when i run docker-compose up anycable_go I get the following

    docker compose up anycable_go
                                                                                                                                                               [789df58]
    [+] Running 3/0
    ✔ Container docker-compose-anycable_go_certs-1  Created                                                                                                                                       0.0s 
    ✔ Container docker-compose-redis-1              Running                                                                                                                                       0.0s 
    ✔ Container docker-compose-anycable_go-1        Created                                                                                                                                       0.0s 
    Attaching to docker-compose-anycable_go-1
    docker-compose-anycable_go-1  | D 2023-12-01T03:49:33.132Z context=main 🔧 🔧 🔧 Debug mode is on 🔧 🔧 🔧
    docker-compose-anycable_go-1  | I 2023-12-01T03:49:33.132Z context=main Starting AnyCable 1.4.6-06129c8 (pid: 1, open file limit: 1048576, gomaxprocs: 6)
    docker-compose-anycable_go-1  | I 2023-12-01T03:49:33.132Z context=main JWT identification is enabled (param: jid, enforced: false)
    docker-compose-anycable_go-1  | I 2023-12-01T03:49:33.132Z context=main Using channels router: Turbo::StreamsChannel
    docker-compose-anycable_go-1  | I 2023-12-01T03:49:33.132Z context=main Anonymized telemetry is on. Learn more: https://docs.anycable.io/anycable-go/telemetry
    docker-compose-anycable_go-1  | I 2023-12-01T03:49:33.132Z
    docker-compose-anycable_go-1  |  context=main Using in-memory broker (epoch: C_VF, history limit: 100, history ttl: 300s, sessions ttl: 300s)
    docker-compose-anycable_go-1  | D 2023-12-01T03:49:33.132Z context=disconnector
    docker-compose-anycable_go-1  |  Calls rate: 10ms         
    docker-compose-anycable_go-1  | I 2023-12-01T03:49:33.132Z context=pubsub Starting Redis pub/sub: redis:6379
    docker-compose-anycable_go-1  | I 2023-12-01T03:49:33.132Z consumer=IygWkh
    docker-compose-anycable_go-1  |  context=broadcast id=IygWkh provider=redisx stream=__anycable__ Starting Redis broadcaster at redis:6379
    docker-compose-anycable_go-1  | I 2023-12-01T03:49:33.133Z context=rpc RPC controller initialized: example.test:50051 (concurrency: 28, impl: grpc, enable_tls: true, proto_versions: v1)
    docker-compose-anycable_go-1  | I 2023-12-01T03:49:33.134Z context=main Handle WebSocket connections at https://example.test8085/cable
    docker-compose-anycable_go-1  | I 2023-12-01T03:49:33.134Z context=main Handle health requests at https://example.test:8085/health
    docker-compose-anycable_go-1  | D 2023-12-01T03:49:33.134Z context=main Go pools initialized (remote commands: 256)
    docker-compose-anycable_go-1  | D 2023-12-01T03:49:33.134Z context=metrics No metrics writers. Disable metrics rotation
    docker-compose-anycable_go-1  | D 2023-12-01T03:49:33.136Z
    docker-compose-anycable_go-1  |  context=http Starting WebSocket server at https://example.test:8085
    docker-compose-anycable_go-1  | D 2023-12-01T03:49:33.139Z channel=__anycable_internal__ context=pubsub Subscribing              
    docker-compose-anycable_go-1  | D 2023-12-01T03:49:33.139Z context=pubsub Subscription message: {subscribe __anycable_internal__ 1}
    docker-compose-anycable_go-1  | D 2023-12-01T03:49:33.139Z channel=__anycable_internal__ context=pubsub Subscribed               
    docker-compose-anycable_go-1  | D 2023-12-01T03:49:33.139Z context=broadcast id=IygWkh provider=redisx Redis consumer group already exists

[palkan]

Let's check the configuration picked up by AnyCable is correct. Can you please show the output of the following command:

$ bin/rails runner "puts AnyCable.config"

...

[omarluq]

bin/rails runner "puts AnyCable.config"                                                                                                                                                                                               [2f477b8689]
#<AnyCable::Config:0x000000011c81df68>

[palkan]

Not this 🙂; let's try on of the below:

bin/rails runner "p AnyCable.config" 

# or
bin/rails runner "p AnyCable.config.to_h"

[omarluq]

bin/rails runner "pp AnyCable.config.to_h"                                                                                                                                                                                            [2f477b8689]
{:presets=>[],
 :broadcast_adapter=>"redis",
 :redis_url=>"redis://redis:6379/0",
 :redis_sentinels=>nil,
 :redis_channel=>"__anycable__",
 :redis_tls_verify=>false,
 :redis_tls_client_cert_path=>nil,
 :redis_tls_client_key_path=>nil,
 :nats_servers=>["nats://localhost:4222"],
 :nats_channel=>"__anycable__",
 :nats_dont_randomize_servers=>false,
 :nats_options=>{},
 :http_broadcast_url=>"http://localhost:8090/_broadcast",
 :http_broadcast_secret=>nil,
 :log_file=>nil,
 :log_level=>"info",
 :debug=>false,
 :http_health_port=>nil,
 :http_health_path=>"/health",
 :http_rpc_secret=>nil,
 :version_check_enabled=>true,
 :sid_header_enabled=>true,
 :rpc_host=>"example.test:50051",
 :rpc_tls_cert=>"/Users/oluqman/Desktop/dev/ue/.certs/anycable-cert.pem",
 :rpc_tls_key=>"/Users/oluqman/Desktop/dev/ue/.certs/anycable-key.pem",
 :rpc_pool_size=>30,
 :rpc_max_waiting_requests=>20,
 :rpc_poll_period=>1,
 :rpc_pool_keep_alive=>0.25,
 :rpc_server_args=>{},
 :rpc_max_connection_age=>300,
 :log_grpc=>true,
 :access_logs_disabled=>false,
 :persistent_session_enabled=>false,
 :embedded=>false,
 :http_rpc_mount_path=>nil,
 :batch_broadcasts=>false,
 :socket_id_header=>"X-Socket-ID",
 :disable_rpc_pool_size_warning=>false,
 :jwt_id_param=>"jid",
 :jwt_id_ttl=>3600,
 :jwt_id_key=>"ue-dev"}

Ahh yes the rpc_tls_cert and rpc_tls_key are pointing to the wrong path, I was using these paths in an earlier config!
Thoughts?

[palkan]

What about pp Anycable.config? (It should show the source of the configuration value, where it comes from).

Check your env variables; probably, they overwrite the defaults

[omarluq]

I was able to fix the error with booting up RPC and both servers are up!
however now I get this on the go server when the frontend tries to connect on load!

docker-compose-anycable_go-1  | 2023/12/01 18:40:27 http: TLS handshake error from 172.18.0.1:49984: remote error: tls: illegal parameter

[omarluq]

I really over complicated this! instead of what I was trying to do initially I rolled back both servers to run without TLS (standard config matching the demo app) and proxied 8085 through puma dev echo 8085 > ~/.puma-dev/anycable_go and for the js client export default createCable('wss://anycable_go.test/cable',anyCableConfig);! problem solved :D