Invalid Auth Error with Home Assistant

[lundyfpv]

I'm sure the issue is on my end but I have tried tons of different methods to get this to work but they all fail with a 401 error in the serial logs. Any idea what I could be doing wrong. I do have my HA instance behind a Cloudflare tunnel but curl seems to work fine for calling services.

Tio Serial Stream:
`I (01:29:23.910) WILLOW: AUDIO_REC_VAD_START
E (01:29:23.911) AUDIO_ELEMENT: [http_stream_writer] Element already stopped
E (01:29:23.912) AUDIO_ELEMENT: [raw_stream_writer_to_api] Element already stopped
I (01:29:23.921) AUDIO_THREAD: The http_stream_writer task allocate stack on external memory
I (01:29:23.931) AUDIO_ELEMENT: [http_stream_writer-0x3d8146a0] Element task created
I (01:29:23.938) AUDIO_ELEMENT: [raw_stream_writer_to_api-0x3d8144b0] Element task created

I (01:29:23.947) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:7128631 Bytes, Inter:135931 Bytes, Dram:135931 Bytes
I (01:29:23.960) AUDIO_ELEMENT: [http_stream_writer] AEL_MSG_CMD_RESUME,state:1
I (01:29:23.968) AUDIO_PIPELINE: Pipeline started
audio_pipeline_run(hdl_ap_to_api) - uri: 'https://infer.tovera.io/api/willow'
I (01:29:23.982) WILLOW: [ + ] HTTP client HTTP_STREAM_PRE_REQUEST, length=0
I (01:29:24.269) WILLOW: AUDIO_REC_VAD_END
I (01:29:24.270) WILLOW: AUDIO_REC_WAKEUP_END
Total bytes written: 4096
Total bytes written: 8192
Total bytes written: 12288
Total bytes written: 16384
Total bytes written: 20480
Total bytes written: 22528
I (01:29:24.674) AUDIO_ELEMENT: IN-[http_stream_writer] AEL_IO_DONE,-2
I (01:29:24.675) WILLOW: [ + ] HTTP client HTTP_STREAM_POST_REQUEST, write end chunked marker
I (01:29:25.059) HTTP_CLIENT: Body received in fetch header state, 0x3d953d63, 31
I (01:29:25.060) WILLOW: [ + ] HTTP client HTTP_STREAM_FINISH_REQUEST
I (01:29:25.064) WILLOW: Got HTTP Response = {"text":"Bye!","language":"en"}
I (01:29:25.072) WILLOW: HASS URL: https://ha.mysite.com:8443/api/conversation/process
I (01:29:25.080) WILLOW: sending '{"text":"Bye!","language":"en"}' to Home Assistant API on 'https://ha.mysite.com:8443/api/conversation/process'
I (01:29:25.919) HTTP_CLIENT: Body received in fetch header state, 0x3d952570, 17
I (01:29:25.920) WILLOW: HTTP status='401' content_length='17'
I (01:29:25.927) AUDIO_THREAD: The play_tone_err task allocate stack on external memory
I (01:29:25.933) WILLOW: HTTP POST response body:
401
I (01:29:27.942) AUDIO_ELEMENT: [http_stream_writer] AEL_MSG_CMD_PAUSE
I (01:29:35.939) WILLOW: Wake LCD timeout, turning off LCD`

HA Logs:
Login attempt or request with invalid authentication from IPaddress (IPaddress). Requested URL: '/api/conversation/process'. (ESP32 HTTP Client/1.0)

And because I was starting to go crazy I also tried using curl with my Long Lived Access Token and that appears to work fine and turn on my input boolean.

curl -X POST -H "Authorization: Bearer 123456......." -H "Content-Type: application/json" -d '{"entity_id":"input_boolean.test"}' "https://ha.mysite.com:8443/api/services/input_boolean/turn_on"

[kristiankielhofner]

Can you provide your curl command with verbose output with the creds removed?

Also, I assume the machine you're running curl on has the WARP client connected to the tunnel? The ESP BOX doesn't have that so for an equivalent test you will need to make sure you have WARP disconnected on the machine you're running curl from.

[lundyfpv]

Thanks for the fast response. I am not running WARP client just using the Cloudflare tunnel for protecting my HA instance.

Verbose Curl Output:

`Note: Unnecessary use of -X or --request, POST is already inferred.

• Trying 104.21.77.241:8443...
• Connected to ha.mysite.com (ipAddress) port 8443 (#0)
• ALPN, offering h2
• ALPN, offering http/1.1
• CAfile: /etc/ssl/certs/ca-certificates.crt
• CApath: /etc/ssl/certs
• TLSv1.0 (OUT), TLS header, Certificate Status (22):
• TLSv1.3 (OUT), TLS handshake, Client hello (1):
• TLSv1.2 (IN), TLS header, Certificate Status (22):
• TLSv1.3 (IN), TLS handshake, Server hello (2):
• TLSv1.2 (IN), TLS header, Finished (20):
• TLSv1.2 (IN), TLS header, Supplemental data (23):
• TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
• TLSv1.3 (IN), TLS handshake, Certificate (11):
• TLSv1.3 (IN), TLS handshake, CERT verify (15):
• TLSv1.3 (IN), TLS handshake, Finished (20):
• TLSv1.2 (OUT), TLS header, Finished (20):
• TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
• TLSv1.2 (OUT), TLS header, Supplemental data (23):
• TLSv1.3 (OUT), TLS handshake, Finished (20):
• SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
• ALPN, server accepted to use h2
• Server certificate:
• subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=sni.cloudflaressl.com
• start date: Jan 1 00:00:00 2023 GMT
• expire date: Jan 1 23:59:59 2024 GMT
• subjectAltName: host "ha.mysite.com" matched cert's "*.mysite.com"
• issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3
• SSL certificate verify ok.
• Using HTTP2, server supports multiplexing
• Connection state changed (HTTP/2 confirmed)
• Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
• TLSv1.2 (OUT), TLS header, Supplemental data (23):
• TLSv1.2 (OUT), TLS header, Supplemental data (23):
• TLSv1.2 (OUT), TLS header, Supplemental data (23):
• Using Stream ID: 1 (easy handle 0xaaab1c3ab040)
• TLSv1.2 (OUT), TLS header, Supplemental data (23):

POST /api/services/input_boolean/turn_on HTTP/2
Host: ha.mysite.com:8443
user-agent: curl/7.81.0
accept: /
authorization: Bearer 111111111111111111111111111111111111111111.......
content-type: application/json
content-length: 34

• TLSv1.2 (OUT), TLS header, Supplemental data (23):
• We are completely uploaded and fine
• TLSv1.2 (IN), TLS header, Supplemental data (23):
• TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
• TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
• old SSL session ID is stale, removing
• TLSv1.2 (IN), TLS header, Supplemental data (23):
• Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
• TLSv1.2 (OUT), TLS header, Supplemental data (23):
• TLSv1.2 (IN), TLS header, Supplemental data (23):
• TLSv1.2 (IN), TLS header, Supplemental data (23):
  < HTTP/2 403
  < date: Tue, 23 May 2023 19:57:35 GMT
  < content-type: text/plain; charset=utf-8
  < content-length: 14
  < cf-cache-status: DYNAMIC
  < report-to: {"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v3?s=Q72lABKWyGRE3eOH%2Fyo%2B%2FZi7rHmKRGX7kYRwg6K6L1nnZMsjcJtJT2QzVyD7cU6C5fUUu9pXlZxzyZdeAQhbeO1uZ9RXXpYEJ%2F3Ipi8iQiMYbYAZ%2BTDMOX8%2F0FFYmblC%2F%2FUaOHsqfg%3D%3D"}],"group":"cf-nel","max_age":604800}
  < nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
  < server: cloudflare
  < cf-ray: 7cbfde6a2c993bd9-MEM
  < alt-svc: h3=":8443"; ma=86400, h3-29=":8443"; ma=86400
  <
• TLSv1.2 (IN), TLS header, Supplemental data (23):
• Connection #0 to host ha.mysite.com left intact`

[kristiankielhofner]

Your example curl is returning 403. Why is that?

I'm also struggling to understand this config (and purpose)... Is your HA instance not local? We've never tested the Willow http client against Cloudflare because frankly we never considered this use case. Theoretically it should "just work" but I have a lot of experience with CF and CF tunnels and I can think of any number of settings/config options that could potentially be problematic.

Can you post a de-authenticated successful curl with -v --http1.1 --tlsv1.2 ?

[lundyfpv]

So might have been a few things. My ip was banned in HA from too many failed auth requests and also apparently I need to add -v at the end of my curl command. So the reason for the Coudflare tunnel is for secure-ish remote access but doing this also prevents local http requests from what I can tell so I have to use my URL that matches my cert. Thanks again for the help though!

Here is an explanation of the Cloudflare tunnel: https://everythingsmarthome.co.uk/free-remote-access-for-home-assistant-with-cloudlfare/

When running: curl -X POST -H "Authorization: Bearer 111111111111111" -H "Content-Type: application/json" -d '{"entity_id":"input_boolean.test"}' "https://ha.mysite.com:8443/api/services/input_boolean/turn_on" -v --http1.1 --tlsv1.2

I get:

• Trying ipAddress:8443...
• Connected to ha.mysite.com (ipAddress) port 8443 (#0)
• ALPN, offering http/1.1
• CAfile: /etc/ssl/certs/ca-certificates.crt
• CApath: /etc/ssl/certs
• TLSv1.0 (OUT), TLS header, Certificate Status (22):
• TLSv1.3 (OUT), TLS handshake, Client hello (1):
• TLSv1.2 (IN), TLS header, Certificate Status (22):
• TLSv1.3 (IN), TLS handshake, Server hello (2):
• TLSv1.2 (IN), TLS header, Finished (20):
• TLSv1.2 (IN), TLS header, Supplemental data (23):
• TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
• TLSv1.3 (IN), TLS handshake, Certificate (11):
• TLSv1.3 (IN), TLS handshake, CERT verify (15):
• TLSv1.3 (IN), TLS handshake, Finished (20):
• TLSv1.2 (OUT), TLS header, Finished (20):
• TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
• TLSv1.2 (OUT), TLS header, Supplemental data (23):
• TLSv1.3 (OUT), TLS handshake, Finished (20):
• SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
• ALPN, server accepted to use http/1.1
• Server certificate:
• subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=sni.cloudflaressl.com
• start date: Jan 1 00:00:00 2023 GMT
• expire date: Jan 1 23:59:59 2024 GMT
• subjectAltName: host "ha.mysite.com" matched cert's "*.mysite.com"
• issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3
• SSL certificate verify ok.
• TLSv1.2 (OUT), TLS header, Supplemental data (23):

POST /api/services/input_boolean/turn_on HTTP/1.1
Host: ha.mysite.com:8443
User-Agent: curl/7.81.0
Accept: /
Authorization: Bearer 111111111111111111111111111111111111
Content-Type: application/json
Content-Length: 34

• TLSv1.2 (IN), TLS header, Supplemental data (23):
• TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
• TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
• old SSL session ID is stale, removing
• TLSv1.2 (IN), TLS header, Supplemental data (23):
• Mark bundle as not supporting multiuse
  < HTTP/1.1 200 OK
  < Date: Tue, 23 May 2023 20:50:26 GMT
  < Content-Type: application/json
  < Transfer-Encoding: chunked
  < Connection: keep-alive
  < CF-Cache-Status: DYNAMIC
  < Report-To: {"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v3?s=uHNPW%2Fg80IO1bYvZopY6RDlAKfEHditN5avyUmQicfcO82QfbadgvADxudPfhgR5eOalE1O%2F2RXHm%2Bm4YX6YM2WS9zo3BPdfWBcoczfIBygBgx%2Fsthq9CeNjH5%2FGHgI3czliAAPhag%3D%3D"}],"group":"cf-nel","max_age":604800}
  < NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
  < Server: cloudflare
  < CF-RAY: 7cc02bd4285d0deb-MEM
  < alt-svc: h3=":8443"; ma=86400, h3-29=":8443"; ma=86400
  <
• TLSv1.2 (IN), TLS header, Supplemental data (23):
• Connection #0 to host ha.mysite.com left intact
  [{"entity_id":"input_boolean.test","state":"on","attributes":{"editable":true,"friendly_name":"test"},"last_changed":"2023-05-23T20:50:26.717845+00:00","last_updated":"2023-05-23T20:50:26.717845+00:00","context":{"id":"01H156188WMRF52760M5Z53T2E","parent_id":null,"user_id":"ad088ed07af74937a14c74482e36811f"}}]

[kristiankielhofner]

What I suspect (more of a desperate hunch) may be happening here is the Willow HTTP client (from ESP-IDF) currently only supports up to TLS 1.2 (TLS 1.3 is in the next release, I believe). Cloudflare allows you to configure TLS 1.3 as the minimum version for the domain:

https://developers.cloudflare.com/ssl/edge-certificates/additional-options/minimum-tls/

Can you check this setting and try setting it to TLS 1.2 and see if that fixes this issue? While you're poking around in the dashboard, make sure you have support for Websockets enabled:

CF Dashboard -> Your domain -> Network -> Websockets

[lundyfpv]

Just want to say thanks again for your help on this, you have gone above and beyond what I expected.

I adjusted my TLS level to 1.2 and disabled 1.3 support and verified websockets are enabled. Still not working sadly, but I did override my local DNS on my router to redirect my URL directly to my Home Assistant IP and that allowed Willow to work, sadly I couldn't access the web interface after doing that so I will have to come up with another solution. Does look like something over the Cloudflare Tunnel is causing issues. I think this is a somewhat popular method of doing remote access to Home Assistant so I will keep digging on my side an report back any solution I am able to find.

[kristiankielhofner]

Thanks!

I suppose what I'm trying to understand most about the use of Cloudflare tunnels with Willow is - why?

Home Assistant's best strength is being locally hosted. Willow is presumably in your home, local, on the same or an immediately adjacent network (VLAN or something). What's the value of communication between Willow and HA going over a CF tunnel? I can understand for access to the HA system when not at home/on the same network, that makes sense. I'm just really not understanding how/where it comes in for Willow-HA communication.

Yes, please do!

[lundyfpv]

You are correct Willow is in my home. It looks like the way I originally configured my Cloudflare tunnel made it where the only way to communicate with it was through the Cloudflare tunnel. I had no local access. I have reconfigured it using the new method below and everything works as expected! Thanks again for battling with me through this weird issue. Got a Tesla P4 on the way to try out WIS!

The 2 methods are below for anyone running into issues. I have included the old method if you need to reverse your steps to get back to square 1.

New Method

Old Method

[kristiankielhofner]

Glad that everything worked out, let us know how things go with WIS!

[hamishcunningham]

I've had good experience with http://pagekite.net/ in the past, might be worth a look.