[SOLVED] Runboat unable to handle webhook when RUNBOAT_GITHUB_WEBHOOK_SECRET not set

[23kbps]

We have runboat up and running in an EKS cluster,
RUNBOAT_GITHUB_WEBHOOK_SECRET is not set in env var
Docker image is ghcr.io/sbidoul/runboat:latest

In repo, webhook is not set the secret. When we push commit, runboat throw exception:

[2022-10-28 03:41:42 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2022-10-28 03:41:42 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2022-10-28 03:41:42 +0000] [1] [INFO] Using worker: runboat.uvicorn.RunboatUvicornWorker
[2022-10-28 03:41:42 +0000] [8] [INFO] Booting worker with pid: 8
[03:41:43] DEBUG    Using selector: EpollSelector [asyncio]                     
[2022-10-28 03:41:43 +0000] [8] [INFO] Started server process [8]
[2022-10-28 03:41:43 +0000] [8] [INFO] Waiting for application startup.
           INFO     Starting controller tasks. [runboat.controller]             
           INFO     (Re)starting deployment_watcher [runboat.controller]        
           INFO     (Re)starting job_watcher [runboat.controller]               
           INFO     (Re)starting cleaner [runboat.controller]                   
           INFO     (Re)starting initializer [runboat.controller]               
           INFO     (Re)starting stopper [runboat.controller]                   
           INFO     (Re)starting undeployer [runboat.controller]                
[2022-10-28 03:41:43 +0000] [8] [INFO] Application startup complete.
           DEBUG    response body:                                              
                    {"kind":"DeploymentList","apiVersion":"apps/v1","metadata":{
                    "resourceVersion":"1193297"},"items":[]}                    
                     [kubernetes.client.rest]                                   
           DEBUG    response body:                                              
                    {"kind":"JobList","apiVersion":"batch/v1","metadata":{"resou
                    rceVersion":"1193297"},"items":[]}                          
                     [kubernetes.client.rest]                                   
10.244.0.9:36040 - "POST /webhooks/github HTTP/1.1" 500
[2022-10-28 03:42:14 +0000] [8] [ERROR] Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 404, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 270, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 75, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 64, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 680, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 275, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 231, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
    return await dependant.call(**values)
  File "/app/runboat/webhooks.py", line 42, in receive_payload
    payload = await request.json()
  File "/usr/local/lib/python3.10/site-packages/starlette/requests.py", line 243, in json
    self._json = json.loads(body)
  File "/usr/local/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

If we set the secret in github webhook and put the secret into k8s secret, runboat just throw a warning and nothing happens.

[2022-10-28 03:44:03 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2022-10-28 03:44:03 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2022-10-28 03:44:03 +0000] [1] [INFO] Using worker: runboat.uvicorn.RunboatUvicornWorker
[2022-10-28 03:44:03 +0000] [6] [INFO] Booting worker with pid: 6
[03:44:04] DEBUG    Using selector: EpollSelector [asyncio]                     
[2022-10-28 03:44:04 +0000] [6] [INFO] Started server process [6]
[2022-10-28 03:44:04 +0000] [6] [INFO] Waiting for application startup.
           INFO     Starting controller tasks. [runboat.controller]             
           INFO     (Re)starting deployment_watcher [runboat.controller]        
           INFO     (Re)starting job_watcher [runboat.controller]               
           INFO     (Re)starting cleaner [runboat.controller]                   
           INFO     (Re)starting initializer [runboat.controller]               
           INFO     (Re)starting stopper [runboat.controller]                   
           INFO     (Re)starting undeployer [runboat.controller]                
[2022-10-28 03:44:04 +0000] [6] [INFO] Application startup complete.
           DEBUG    response body:                                              
                    {"kind":"JobList","apiVersion":"batch/v1","metadata":{"resou
                    rceVersion":"1193900"},"items":[]}                          
                     [kubernetes.client.rest]                                   
           DEBUG    response body:                                              
                    {"kind":"DeploymentList","apiVersion":"apps/v1","metadata":{
                    "resourceVersion":"1193900"},"items":[]}                    
                     [kubernetes.client.rest]                                   
[03:44:57] WARNING  Got payload with invalid X-Hub-Signature-256                
                    [runboat.webhooks]                                          
10.244.0.9:33604 - "POST /webhooks/github HTTP/1.1" 200

I can confirm that webhook secret and k8s secret is setup correctly.

This is our new build and first time using runboat, any idea that I could do it wrong in any step?

Thanks

[sbidoul]

Your first stack trace is possibly due to the webhook being configured to post x-www-url-encoded instead of json.
The second warning "Got payload with invalid X-Hub-Signature-256" is likely a configuration issue with the secret. Make sure your runboat pod has a RUNBOAT_GITHUB_WEBHOOK_SECRET and it matches the value configured in the GitHub webhook.
Or it is a bug, but this feature is known to work in the OCA deployment.

[23kbps]

Thanks @sbidoul the content type was not configured properly, switched to json and all works fine.
Another question, I can't find how to delete a build to completely destroy everything related of that build, how can I do that?

[sbidoul]

You can delete the corresponding deployment ressource with kubectl delete and runboat will cleanup everything related to it. Or use the runboat REST API.

[23kbps]

Nice thanks, this is great tool!