Support upstream timeouts for Gateway API

[Alice-Lilith]

Description

HTTPRoute.Spec.Rules.Timeouts currently cannot be supported because there is no way (as of the time of writing this) to configure upstream or total request timeouts with ngrok.

type HTTPRouteTimeouts struct {
    // Request specifies the maximum duration for a gateway to respond to an HTTP request.
    // If the gateway has not been able to respond before this deadline is met, the gateway
    // MUST return a timeout error.
    //
    // For example, setting the `rules.timeouts.request` field to the value `10s` in an
    // `HTTPRoute` will cause a timeout if a client request is taking longer than 10 seconds
    // to complete.
    //
    // This timeout is intended to cover as close to the whole request-response transaction
    // as possible although an implementation MAY choose to start the timeout after the entire
    // request stream has been received instead of immediately after the transaction is
    // initiated by the client.
    //
    // When this field is unspecified, request timeout behavior is implementation-specific.
    //
    // Support: Extended
    //
    // +optional
    Request *Duration `json:"request,omitempty"`

    // BackendRequest specifies a timeout for an individual request from the gateway
    // to a backend. This covers the time from when the request first starts being
    // sent from the gateway to when the full response has been received from the backend.
    //
    // An entire client HTTP transaction with a gateway, covered by the Request timeout,
    // may result in more than one call from the gateway to the destination backend,
    // for example, if automatic retries are supported.
    //
    // Because the Request timeout encompasses the BackendRequest timeout, the value of
    // BackendRequest must be <= the value of Request timeout.
    //
    // Support: Extended
    //
    // +optional
    BackendRequest *Duration `json:"backendRequest,omitempty"`
}

We could potentially make a timeout configuration for CloudEndpoints so that you can configure the total timeout for requests to that CloudEndpoint, but it would leave us in a tricky situation for Gateway API support, since this configuration could be different for different backends. Perhaps something like a new timeout traffic policy action that once fired, the traffic policy will continue running subsequent actions, but terminate early if any timeouts set by prior timeout actions are triggered.

If we were to go that route, you could create a traffic policy like a configuration similar to the following to set a timeout for any requests to a cloud endpoint

    on_http_request:
      - name: global-timeout
        actions:
          - type: timeout
            config:
              id: "global-timeout" # perhaps something like an id field would help manage different timeouts such as overriding/stacking them
              duration: "3s"
      - actions:
        - type: forward-internal
           config:
             url: https://some-service.internal

Or if you wanted to set a more lenient/strict timeout for certain upstreams you could maybe do something like...

    on_http_request:
      - name: global-timeout
        actions:
          - type: timeout
            config:
              id: "global-timeout"
              duration: "3s"
      - actions:
        - name: route-timeout
          actions:
            - type: timeout
              config:
                id: "global-timeout" # override the global timeout for this service we expect to take a little longer (if you wanted a shorter timeout you could make a new ID and that one would fire before the global timeout did)
                duration: "5s"
        - name: slow-upstream-needs-more-time
          type: forward-internal
           config:
             url: https://some-service.internal

These are just some rough thoughts about what supporting these timeouts with the ngrok feature set might look like. I imagine an actual implementation would need a lot more thought put into it such as potentially allowing you to fire other traffic policy actions when the timeout does occur or simply setting the content/headers sent when a timeout fires.