Local Service Chaining and Intra-App routing not working with selective deployment

[mikkelhegn]

Trying to deploy a Spin application with two components, using local service chaining, the component calling another component fails with the following message:

Error with selective deployment: Component selected with '--component frontend' cannot use service chaining to unselected component: allowed_outbound_hosts = ["http://backend.spin.internal"]

Using the Intra-app method "http://self" also does not work, it returns 404 trying to route to another component.

It would be nice to see service chaining working across components with selective deployment.

[mikkelhegn]

How did I test this given spinframework/containerd-shim-spin#238 - one may ask oneself :-)

I was provided a build from this fork with support for selective deployment: radu-matei/tmp-containerd-shim-spin@6c91229

Edited: never mind this comment, I confused myself for a moment here.

[endocrimes]

I think this needs to be an issue in Spin - there's no way (afaik) to tell it what it should do for unselected components.

The service chaining contract doesn't let those requests suddenly become remote at the moment, so it might not be solvable as-is.

For the self path, depending on how the shim resolves self, you could configure an ingress (which is unmanaged by SpinKube https://www.spinkube.dev/docs/topics/routing/) with multiple paths (https://kubernetes.io/docs/concepts/services-networking/ingress/#the-ingress-resource) and that could potentially work

[mikkelhegn]

I think this needs to be an issue in Spin

It's a good point that Spin should probably make up it's mind of what's the appropriate way of doing component-to-component communication, given selective deployments is a thing (local Service Chaining or "self"). E.g., "If you do this, it's guaranteed to work.".

...there's no way (afaik) to tell it what it should do for unselected components.

spin.toml does provide hints of which components a given component wants to reach, at least in the case of local service chaining. Whether it's deployed in the cluster or somewhere else is not hinted. My expectation of the spirit of the feature is that I should be able to not care. I guess there is a design missing for how to provide some kind of configuration to help Spinkube set up the routing?

[kate-goldenring]

@mikkelhegn, as you found, support service chaining and self requests is limited for selective deployment. This is documented in Spin's docs as that is the initial scope of support: https://developer.fermyon.com/spin/v3/running-apps#splitting-an-application-across-environments

One workaround is for Spin/SpinKube users to set the route to other deployments as spin application variables. For example, say in the salutations app, you update the hello component to call the goodbye component and append the contents to the HTTP response (effectively returning "hello goodbye"). You could do the following:

Add the goodbye address as an application variable in allowed hosts (and update the call in the implementation to use the application variable):

[component.hello]
source = "../hello-world/main.wasm"
allowed_outbound_hosts = ["{goodbye-route}"]
[component.hello.build]
command = "cd ../hello-world && tinygo build -target=wasi -gc=leaking -no-debug -o main.wasm main.go"
watch = ["**/*.go", "go.mod"]

Add the application variable to the SpinApp:

apiVersion: core.spinkube.dev/v1alpha1
kind: SpinApp
metadata:
  name: hello-spinapp
spec:
  image: "ghcr.io/spinkube/spin-operator/salutations:20241105-223428-g4da3171"
  replicas: 1
  executor: containerd-shim-spin
  components: ["hello"]
  variables:
    - name: goodbye-route
      value: http://goodbye-spinapp.default.svc.cluster.local

---
apiVersion: core.spinkube.dev/v1alpha1
kind: SpinApp
metadata:
  name: goodbye-spinapp
spec:
  image: "ghcr.io/spinkube/spin-operator/salutations:20241105-223428-g4da3171"
  replicas: 1
  executor: containerd-shim-spin
  components: ["goodbye"]

Right now, the Spin Operator never looks at a Spin.toml file and has no understanding of which components are trying to communicate. If we want to enable self requests with selective deployment, the operator would likely need to open the OCI artifacts and keep an understanding of where each component has been deployed. We'd need to implement in spin a new way to map self requests too.

[mikkelhegn]

Thanks for all the details @kate-goldenring, and the workaround of providing addresses via config. I still think it would be great if there was an easy solution, that doesn't require config specific to the environment, to have two Spin components communicate, also when using selective deployments.

@tschneidereit and I discussed this earlier today "in-person", and one idea that came up was to check for the use of Local Service Chaining or "self" at deployment time to block those components which depend on others to be selectively deployed. At least to help users fail fast.